2020-11-14T02:13:18.505 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-14T02:13:18.507 INFO:teuthology.results:subset: '0/4' 2020-11-14T02:13:18.507 INFO:teuthology.results:seed: '1265' 2020-11-14T15:29:19.458 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-14T15:29:19.504 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-11-15T03:22:44.596 INFO:teuthology.results:Tests finished! gathering results... 2020-11-15T03:23:42.506 INFO:scrape:Found 335 jobs 2020-11-15T03:24:08.263 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620817/teuthology.log matches 2020-11-15T03:24:08.264 WARNING:scrape:Not-understood crash indication 2020-11-15T03:24:08.319 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620817/teuthology.log matches 2020-11-15T03:24:08.320 WARNING:scrape:Not-understood crash indication 2020-11-15T03:24:08.371 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620817/teuthology.log matches 2020-11-15T03:24:08.372 WARNING:scrape:Not-understood crash indication 2020-11-15T03:24:08.428 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620817/teuthology.log matches 2020-11-15T03:24:08.429 WARNING:scrape:Not-understood crash indication 2020-11-15T03:24:08.483 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620817/teuthology.log matches 2020-11-15T03:24:08.484 WARNING:scrape:Not-understood crash indication 2020-11-15T03:24:08.534 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620817/teuthology.log matches 2020-11-15T03:24:08.535 WARNING:scrape:Not-understood crash indication 2020-11-15T03:24:08.585 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620817/teuthology.log matches 2020-11-15T03:24:08.586 WARNING:scrape:Not-understood crash indication 2020-11-15T03:24:08.638 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620817/teuthology.log matches 2020-11-15T03:24:08.639 WARNING:scrape:Not-understood crash indication 2020-11-15T03:24:08.692 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620817/teuthology.log matches 2020-11-15T03:24:08.693 WARNING:scrape:Not-understood crash indication 2020-11-15T03:24:08.755 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620817/teuthology.log matches 2020-11-15T03:24:08.756 WARNING:scrape:Not-understood crash indication 2020-11-15T03:24:08.807 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620817/teuthology.log matches 2020-11-15T03:24:08.808 WARNING:scrape:Not-understood crash indication 2020-11-15T03:24:53.634 INFO:scrape:Found 16 distinct failure reasons 2020-11-15T03:24:53.636 INFO:scrape:Crash: Test failure: test_15303 (tasks.cephfs.test_volume_client.TestVolumeClient) 2020-11-15T03:24:53.636 INFO:scrape:ceph version 16.0.0-7160-g4090b2dd (4090b2ddb3668718593e366f72a099e049d8e78a) pacific (dev) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7f9480ec2980] 2: gsignal() 3: abort() 4: /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8c957) [0x7f94809af957] 5: /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x92ae6) [0x7f94809b5ae6] 6: /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x92b21) [0x7f94809b5b21] 7: /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x92d54) [0x7f94809b5d54] 8: /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8e837) [0x7f94809b1837] 9: ceph-fuse(+0x13001e) [0x561a070b501e] 10: (Client::_create(Inode*, char const*, int, unsigned int, boost::intrusive_ptr*, Fh**, int, int, int, char const*, bool*, UserPerm const&)+0x4bb) [0x561a0706a31b] 11: (Client::_ll_create(Inode*, char const*, unsigned int, int, boost::intrusive_ptr*, int, Fh**, UserPerm const&)+0x21d) [0x561a0706b7ed] 12: (Client::ll_create(Inode*, char const*, unsigned int, int, stat*, Inode**, Fh**, UserPerm const&)+0x110) [0x561a0706c180] 13: ceph-fuse(+0x69dcb) [0x561a06feedcb] 14: /lib/x86_64-linux-gnu/libfuse.so.2(+0x13cbd) [0x7f948a198cbd] 15: /lib/x86_64-linux-gnu/libfuse.so.2(+0x156c1) [0x7f948a19a6c1] 16: /lib/x86_64-linux-gnu/libfuse.so.2(+0x11e68) [0x7f948a196e68] 17: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f9480eb76db] 18: clone() 2020-11-15T03:24:53.637 INFO:scrape:56 jobs 2020-11-15T03:24:53.638 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.638 INFO:scrape:suites union: ['1-client', '2-upgrade', '2-workunit/cfuse_workunit_snaptests}}', '2-workunit/cfuse_workunit_suites_pjd}}', '2-workunit/cfuse_workunit_trivial_sync}}', '2-workunit/ffsb}}', '2-workunit/iozone}}', '3-client-upgrade', '4-compat_client', '5-client-sanity}}', 'clusters/1-mds-1-client-coloc', 'clusters/1-mds-2-client', 'clusters/1-mds-2-client-micro', 'clusters/1-mds-3-client', 'clusters/1-mds-4-client-coloc', 'clusters/1a2s-mds-1c-client-3node', 'clusters/1a3s-mds-1c-client', 'clusters/1a3s-mds-2c-client', 'clusters/1a3s-mds-4c-client', 'clusters/fixed-2-ucephfs', 'conf/{client', 'distro/{ubuntu_latest}', 'distros/ubuntu_latest', 'frag_enable', 'fs/32bits/{begin', 'fs/functional/{begin', 'fs/mixed-clients/{begin', 'fs/multiclient/{begin', 'fs/multifs/{begin', 'fs/permission/{begin', 'fs/snaps/{begin', 'fs/thrash/{begin', 'fs/traceless/{begin', 'fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap', 'fs/volumes/{begin', 'fs/workload/{begin', 'k-testing}', 'kclient-overrides/{distro/testing/{flavor/ubuntu_latest', 'mds', 'mon', 'mon-debug', 'mount/fuse', 'ms-die-on-skipped}', 'msgr-failures/none', 'msgr-failures/osd-mds-delay', 'multimds/no', 'multimds/yes', 'no_client_pidfile', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore-ec/filestore-xfs', 'objectstore/bluestore-bitmap', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{faked-ino', 'overrides/{frag_enable', 'pg-warn', 'session_timeout', 'tasks/admin}', 'tasks/backtrace}', 'tasks/cephfs_misc_tests}', 'tasks/cfuse_workunit_suites_blogbench', 'tasks/cfuse_workunit_suites_dbench', 'tasks/cfuse_workunit_suites_ffsb', 'tasks/cfuse_workunit_suites_fsstress}', 'tasks/cfuse_workunit_suites_pjd}', 'tasks/failover}', 'tasks/ior-shared-file}', 'tasks/kernel_cfuse_workunits_dbench_iozone}', 'tasks/mds-full}', 'tasks/truncate_delay}', 'tasks/volume-client}', 'tasks/workunit/direct_io}', 'tasks/workunit/fs/norstats}', 'tasks/workunit/fs/test_o_trunc}', 'tasks/workunit/kernel_untar_build}', 'tasks/workunit/snaps}', 'tasks/workunit/suites/dbench}', 'tasks/workunit/suites/iozone}', 'tasks/{0-nautilus', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'tasks/{1-thrash/osd', 'thrashosds-health', 'traceless/50pc}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.639 INFO:scrape: 2020-11-15T03:24:53.639 INFO:scrape:Crash: Command failed (workunit test suites/dbench.sh) on smithi063 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2020-11-15T03:24:53.639 INFO:scrape:ceph version 16.0.0-7160-g4090b2dd (4090b2ddb3668718593e366f72a099e049d8e78a) pacific (dev) 1: /lib64/libpthread.so.0(+0x12dc0) [0x7feb05a5adc0] 2: gsignal() 3: abort() 4: /lib64/libstdc++.so.6(+0x9006b) [0x7feb04ed406b] 5: /lib64/libstdc++.so.6(+0x964cc) [0x7feb04eda4cc] 6: /lib64/libstdc++.so.6(+0x96527) [0x7feb04eda527] 7: /lib64/libstdc++.so.6(+0x96788) [0x7feb04eda788] 8: /lib64/libstdc++.so.6(+0x9200f) [0x7feb04ed600f] 9: (Inode::get_open_ref(int)+0x13c) [0x55693035ea9c] 10: (Client::_create(Inode*, char const*, int, unsigned int, boost::intrusive_ptr*, Fh**, int, int, int, char const*, bool*, UserPerm const&)+0x452) [0x55693030aae2] 11: (Client::_ll_create(Inode*, char const*, unsigned int, int, boost::intrusive_ptr*, int, Fh**, UserPerm const&)+0x219) [0x55693030c089] 12: (Client::ll_create(Inode*, char const*, unsigned int, int, stat*, Inode**, Fh**, UserPerm const&)+0xc5) [0x55693030ca35] 13: ceph-fuse(+0x78a36) [0x556930299a36] 14: /lib64/libfuse.so.2(+0x162ab) [0x7feb0fad02ab] 15: /lib64/libfuse.so.2(+0x17818) [0x7feb0fad1818] 16: /lib64/libfuse.so.2(+0x14400) [0x7feb0face400] 17: /lib64/libpthread.so.0(+0x82de) [0x7feb05a502de] 18: clone() 2020-11-15T03:24:53.640 INFO:scrape:85 jobs 2020-11-15T03:24:53.641 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.641 INFO:scrape:suites union: ['2-workunit/cfuse_workunit_snaptests}}', '2-workunit/cfuse_workunit_suites_fsstress}}', '2-workunit/cfuse_workunit_suites_pjd}}', '2-workunit/cfuse_workunit_trivial_sync}}', '2-workunit/ffsb}}', '2-workunit/iozone}}', 'centos_latest', 'clusters/1-mds-1-client-coloc', 'clusters/1-mds-4-client-coloc', 'clusters/1a2s-mds-1c-client-3node', 'clusters/1a3s-mds-1c-client', 'clusters/1a3s-mds-2c-client', 'clusters/fixed-2-ucephfs', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'frag_enable', 'fs/32bits/{begin', 'fs/functional/{begin', 'fs/mixed-clients/{begin', 'fs/multifs/{begin', 'fs/permission/{begin', 'fs/snaps/{begin', 'fs/thrash/{begin', 'fs/traceless/{begin', 'fs/verify/{begin', 'fs/workload/{begin', 'k-testing}', 'kclient-overrides/{distro/stock/{k-stock', 'kclient-overrides/{distro/testing/{flavor/centos_latest', 'mds', 'mon', 'mon-debug', 'mount/fuse', 'ms-die-on-skipped}', 'msgr-failures/none', 'msgr-failures/osd-mds-delay', 'no_client_pidfile', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore-ec/filestore-xfs', 'objectstore/bluestore-bitmap', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{faked-ino', 'overrides/{frag_enable', 'rhel_8}', 'session_timeout', 'tasks/auto-repair}', 'tasks/cfuse_workunit_suites_blogbench', 'tasks/cfuse_workunit_suites_dbench', 'tasks/cfuse_workunit_suites_ffsb', 'tasks/cfuse_workunit_suites_fsstress', 'tasks/cfuse_workunit_suites_fsstress}', 'tasks/cfuse_workunit_suites_pjd}', 'tasks/client-readahead}', 'tasks/client-recovery}', 'tasks/failover}', 'tasks/forward-scrub}', 'tasks/fragment}', 'tasks/kernel_cfuse_workunits_dbench_iozone}', 'tasks/quota}', 'tasks/scrub}', 'tasks/test_journal_migration}', 'tasks/workunit/direct_io}', 'tasks/workunit/fs/norstats}', 'tasks/workunit/fs/test_o_trunc}', 'tasks/workunit/kernel_untar_build}', 'tasks/workunit/snaps}', 'tasks/workunit/suites/dbench}', 'tasks/workunit/suites/fsx}', 'tasks/workunit/suites/iozone}', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'tasks/{1-thrash/osd', 'thrashosds-health', 'traceless/50pc}', 'validater/lockdep}', 'validater/valgrind}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.641 INFO:scrape: 2020-11-15T03:24:53.642 INFO:scrape:Crash: Test failure: test_rebuild_simple (tasks.cephfs.test_recovery_pool.TestRecoveryPool) 2020-11-15T03:24:53.642 INFO:scrape:ceph version 16.0.0-7160-g4090b2dd (4090b2ddb3668718593e366f72a099e049d8e78a) pacific (dev) 1: /lib64/libpthread.so.0(+0x12dc0) [0x7f8fb23d6dc0] 2: (operator<<(std::ostream&, CDentry const&)+0x6d5) [0x55ee905bdf85] 3: (MDCache::_create_system_file_finish(boost::intrusive_ptr&, CDentry*, unsigned long, MDSContext*)+0xcd) [0x55ee90472d3d] 4: (MDSContext::complete(int)+0x56) [0x55ee90682306] 5: (MDSIOContextBase::complete(int)+0xa3) [0x55ee90682633] 6: (MDSLogContextBase::complete(int)+0x44) [0x55ee90682d84] 7: (Finisher::finisher_thread_entry()+0x1a5) [0x7f8fb3ab7015] 8: /lib64/libpthread.so.0(+0x82de) [0x7f8fb23cc2de] 9: clone() 2020-11-15T03:24:53.642 INFO:scrape:1 jobs: ['5620662'] 2020-11-15T03:24:53.643 INFO:scrape:suites: ['clusters/1-mds-4-client-coloc', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/alternate-pool}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.643 INFO:scrape: 2020-11-15T03:24:53.643 INFO:scrape:Crash: Command failed (workunit test fs/misc/acl.sh) on smithi172 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh' 2020-11-15T03:24:53.644 INFO:scrape:ceph version 16.0.0-7160-g4090b2dd (4090b2ddb3668718593e366f72a099e049d8e78a) pacific (dev) 1: /lib64/libpthread.so.0(+0x12dc0) [0x7f07febbadc0] 2: (Client::flush_cap_releases()+0x13f) [0x55735172ab4f] 3: (Client::renew_and_flush_cap_releases()+0x11c) [0x557351781f0c] 4: (Client::tick()+0x176) [0x5573517838d6] 5: ceph-fuse(+0x101f55) [0x557351783f55] 6: /lib64/libstdc++.so.6(+0xc2b23) [0x7f07fe061b23] 7: /lib64/libpthread.so.0(+0x82de) [0x7f07febb02de] 8: clone() 2020-11-15T03:24:53.644 INFO:scrape:3 jobs: ['5620646', '5620712', '5620913'] 2020-11-15T03:24:53.645 INFO:scrape:suites intersection: ['clusters/fixed-2-ucephfs', 'conf/{client', 'fs/permission/{begin', 'mds', 'mon', 'mount/fuse', 'osd}', 'overrides/{frag_enable', 'tasks/cfuse_workunit_misc}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.645 INFO:scrape:suites union: ['clusters/fixed-2-ucephfs', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'fs/permission/{begin', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{frag_enable', 'tasks/cfuse_workunit_misc}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.645 INFO:scrape: 2020-11-15T03:24:53.646 INFO:scrape:Dead: b'2020-11-14T14:40:06.196 DEBUG:teuthology.exit:Finished running handlers' 2020-11-15T03:24:53.646 INFO:scrape:ceph version 16.0.0-7160-g4090b2dd (4090b2ddb3668718593e366f72a099e049d8e78a) pacific (dev) 1: /lib64/libpthread.so.0(+0x12dc0) [0x7f2b6c4bedc0] 2: gsignal() 3: abort() 4: /lib64/libstdc++.so.6(+0x9006b) [0x7f2b6b93806b] 5: /lib64/libstdc++.so.6(+0x964cc) [0x7f2b6b93e4cc] 6: /lib64/libstdc++.so.6(+0x96527) [0x7f2b6b93e527] 7: /lib64/libstdc++.so.6(+0x96788) [0x7f2b6b93e788] 8: /lib64/libstdc++.so.6(+0x9200f) [0x7f2b6b93a00f] 9: (Inode::get_open_ref(int)+0x13c) [0x55814dd3da9c] 10: (Client::_create(Inode*, char const*, int, unsigned int, boost::intrusive_ptr*, Fh**, int, int, int, char const*, bool*, UserPerm const&)+0x452) [0x55814dce9ae2] 11: (Client::_ll_create(Inode*, char const*, unsigned int, int, boost::intrusive_ptr*, int, Fh**, UserPerm const&)+0x219) [0x55814dceb089] 12: (Client::ll_create(Inode*, char const*, unsigned int, int, stat*, Inode**, Fh**, UserPerm const&)+0xc5) [0x55814dceba35] 13: ceph-fuse(+0x78a36) [0x55814dc78a36] 14: /lib64/libfuse.so.2(+0x162ab) [0x7f2b765342ab] 15: /lib64/libfuse.so.2(+0x17818) [0x7f2b76535818] 16: /lib64/libfuse.so.2(+0x14400) [0x7f2b76532400] 17: /lib64/libpthread.so.0(+0x82de) [0x7f2b6c4b42de] 18: clone() 2020-11-15T03:24:53.646 INFO:scrape:3 jobs: ['5620692', '5620776', '5620958'] 2020-11-15T03:24:53.647 INFO:scrape:suites intersection: ['clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin', 'mds', 'mon', 'ms-die-on-skipped}', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.647 INFO:scrape:suites union: ['clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin', 'k-testing}', 'kclient-overrides/{distro/stock/{k-stock', 'kclient-overrides/{distro/testing/{flavor/centos_latest', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'rhel_8}', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.647 INFO:scrape: 2020-11-15T03:24:53.648 INFO:scrape:Crash: Test failure: test_clone_subvolume_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2020-11-15T03:24:53.648 INFO:scrape:ceph version 16.0.0-7160-g4090b2dd (4090b2ddb3668718593e366f72a099e049d8e78a) pacific (dev) 1: /lib64/libpthread.so.0(+0x12dc0) [0x7f6539225dc0] 2: gsignal() 3: abort() 4: /lib64/libstdc++.so.6(+0x9006b) [0x7f653869f06b] 5: /lib64/libstdc++.so.6(+0x964cc) [0x7f65386a54cc] 6: /lib64/libstdc++.so.6(+0x96527) [0x7f65386a5527] 7: /lib64/libstdc++.so.6(+0x96788) [0x7f65386a5788] 8: /lib64/libstdc++.so.6(+0x9200f) [0x7f65386a100f] 9: /lib64/libcephfs.so.2(+0x1024dc) [0x7f652a7ce4dc] 10: /lib64/libcephfs.so.2(+0xae442) [0x7f652a77a442] 11: /lib64/libcephfs.so.2(+0xaf46a) [0x7f652a77b46a] 12: /lib64/libcephfs.so.2(+0xaf7c6) [0x7f652a77b7c6] 13: /usr/lib64/python3.6/site-packages/cephfs.cpython-36m-x86_64-linux-gnu.so(+0x58b43) [0x7f652aa91b43] 14: _PyCFunction_FastCallDict() 15: /lib64/libpython3.6m.so.1.0(+0x17f94d) [0x7f653a73f94d] 16: _PyEval_EvalFrameDefault() 17: /lib64/libpython3.6m.so.1.0(+0x13b42a) [0x7f653a6fb42a] 18: /lib64/libpython3.6m.so.1.0(+0x17fa32) [0x7f653a73fa32] 19: _PyEval_EvalFrameDefault() 20: /lib64/libpython3.6m.so.1.0(+0x13b42a) [0x7f653a6fb42a] 21: /lib64/libpython3.6m.so.1.0(+0x17fa32) [0x7f653a73fa32] 22: _PyEval_EvalFrameDefault() 23: /lib64/libpython3.6m.so.1.0(+0x13b42a) [0x7f653a6fb42a] 24: /lib64/libpython3.6m.so.1.0(+0x17fa32) [0x7f653a73fa32] 25: _PyEval_EvalFrameDefault() 26: /lib64/libpython3.6m.so.1.0(+0x13b42a) [0x7f653a6fb42a] 27: /lib64/libpython3.6m.so.1.0(+0x17fa32) [0x7f653a73fa32] 28: _PyEval_EvalFrameDefault() 29: /lib64/libpython3.6m.so.1.0(+0x101be8) [0x7f653a6c1be8] 30: _PyFunction_FastCallDict() 31: _PyObject_FastCallDict() 32: _PyObject_Call_Prepend() 33: PyObject_Call() 34: _PyEval_EvalFrameDefault() 35: /lib64/libpython3.6m.so.1.0(+0x101be8) [0x7f653a6c1be8] 36: /lib64/libpython3.6m.so.1.0(+0x13b5e0) [0x7f653a6fb5e0] 37: /lib64/libpython3.6m.so.1.0(+0x17fa32) [0x7f653a73fa32] 38: _PyEval_EvalFrameDefault() 39: /lib64/libpython3.6m.so.1.0(+0x13b42a) [0x7f653a6fb42a] 40: /lib64/libpython3.6m.so.1.0(+0x17fa32) [0x7f653a73fa32] 41: _PyEval_EvalFrameDefault() 42: /lib64/libpython3.6m.so.1.0(+0x101be8) [0x7f653a6c1be8] 43: /lib64/libpython3.6m.so.1.0(+0x13b5e0) [0x7f653a6fb5e0] 44: /lib64/libpython3.6m.so.1.0(+0x17fa32) [0x7f653a73fa32] 45: _PyEval_EvalFrameDefault() 46: /lib64/libpython3.6m.so.1.0(+0x13b42a) [0x7f653a6fb42a] 47: /lib64/libpython3.6m.so.1.0(+0x17fa32) [0x7f653a73fa32] 48: _PyEval_EvalFrameDefault() 49: _PyFunction_FastCallDict() 50: _PyObject_FastCallDict() 51: _PyObject_Call_Prepend() 52: PyObject_Call() 53: PyObject_CallMethod() 54: (ActivePyModule::handle_command(ModuleCommand const&, MgrSession const&, std::map, std::allocator >, boost::variant, std::allocator >, bool, long, double, std::vector, std::allocator >, std::allocator, std::allocator > > >, std::vector >, std::vector > >, std::less, std::allocator, std::allocator > const, boost::variant, std::allocator >, bool, long, double, std::vector, std::allocator >, std::allocator, std::allocator > > >, std::vector >, std::vector > > > > > const&, ceph::buffer::v15_2_0::list const&, std::__cxx11::basic_stringstream, std::allocator >*, std::__cxx11::basic_stringstream, std::allocator >*)+0x222) [0x559c02f7cd12] 55: ceph-mgr(+0x1f5b4d) [0x559c02ff5b4d] 56: (Context::complete(int)+0xd) [0x559c02f8d3ed] 57: (Finisher::finisher_thread_entry()+0x1a5) [0x7f653ae28015] 58: /lib64/libpthread.so.0(+0x82de) [0x7f653921b2de] 59: clone() 2020-11-15T03:24:53.648 INFO:scrape:5 jobs: ['5620717', '5620751', '5620696', '5620883', '5620918'] 2020-11-15T03:24:53.649 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'overrides/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.649 INFO:scrape:suites union: ['1-client', '2-upgrade', '3-verify}', 'clusters/1-mds-2-client-micro', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/upgrade/volumes/import-legacy/{bluestore-bitmap', 'fs/volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'pg-warn', 'tasks/volumes/{overrides', 'tasks/{0-nautilus', 'test/clone}}', 'test/misc}}', 'ubuntu_18.04}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.649 INFO:scrape: 2020-11-15T03:24:53.650 INFO:scrape:Failure: Test failure: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2020-11-15T03:24:53.650 INFO:scrape:1 jobs: ['5620908'] 2020-11-15T03:24:53.650 INFO:scrape:suites: ['clusters/1-mds-4-client-coloc', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/mds-full}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.651 INFO:scrape: 2020-11-15T03:24:53.651 INFO:scrape:Crash: Command failed on smithi196 with status 1: 'mpiexec -f /home/ubuntu/cephtest/mpi-hosts /home/ubuntu/cephtest/mdtest-1.9.3/mdtest -d /home/ubuntu/cephtest/gmnt -I 20 -z 5 -b 2 -R' 2020-11-15T03:24:53.651 INFO:scrape:ceph version 16.0.0-7160-g4090b2dd (4090b2ddb3668718593e366f72a099e049d8e78a) pacific (dev) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7fb0c8d13980] 2: gsignal() 3: abort() 4: /lib/x86_64-linux-gnu/libc.so.6(+0x3048a) [0x7fb0c7dfd48a] 5: /lib/x86_64-linux-gnu/libc.so.6(+0x30502) [0x7fb0c7dfd502] 6: (Client::flush_cap_releases()+0x21d) [0x55b3ca0e696d] 7: (Client::renew_and_flush_cap_releases()+0x11c) [0x55b3ca0e7a9c] 8: (Client::tick()+0x9d) [0x55b3ca0fef5d] 9: ceph-fuse(+0xb3708) [0x55b3ca0ff708] 10: /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xbd6df) [0x7fb0c88316df] 11: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fb0c8d086db] 12: clone() 2020-11-15T03:24:53.652 INFO:scrape:5 jobs: ['5620869', '5620779', '5620846', '5620784', '5620735'] 2020-11-15T03:24:53.652 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'mount/fuse', 'osd}', 'overrides/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.652 INFO:scrape:suites union: ['clusters/1-mds-2-client', 'clusters/1-mds-3-client', 'clusters/1a3s-mds-4c-client', 'clusters/fixed-2-ucephfs', 'conf/{client', 'distro/{ubuntu_latest}', 'distros/ubuntu_latest', 'fs/multiclient/{begin', 'fs/permission/{begin', 'fs/volumes/{begin', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/filestore-xfs', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/cfuse_workunit_misc}', 'tasks/mdtest}', 'tasks/volumes/{overrides', 'test/snapshot}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.653 INFO:scrape: 2020-11-15T03:24:53.653 INFO:scrape:Crash: Test failure: test_subvolume_group_create_with_desired_data_pool_layout (tasks.cephfs.test_volumes.TestSubvolumeGroups) 2020-11-15T03:24:53.654 INFO:scrape:ceph version 16.0.0-7160-g4090b2dd (4090b2ddb3668718593e366f72a099e049d8e78a) pacific (dev) 1: /lib64/libpthread.so.0(+0x12dc0) [0x7fa657fa3dc0] 2: /lib64/libcephfs.so.2(+0x6d4af) [0x7fa6495334af] 3: /lib64/libcephfs.so.2(+0xc486c) [0x7fa64958a86c] 4: /lib64/libcephfs.so.2(+0xc6236) [0x7fa64958c236] 5: /lib64/libcephfs.so.2(+0xc68b5) [0x7fa64958c8b5] 6: /lib64/libstdc++.so.6(+0xc2b23) [0x7fa65744ab23] 7: /lib64/libpthread.so.0(+0x82de) [0x7fa657f992de] 8: clone() 2020-11-15T03:24:53.654 INFO:scrape:3 jobs: ['5620683', '5620851', '5620949'] 2020-11-15T03:24:53.654 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/volumes/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.655 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'rhel_8}', 'tasks/volumes/{overrides', 'test/basic}}', 'test/snapshot}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.655 INFO:scrape: 2020-11-15T03:24:53.655 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/suites/iozone.sh 2020-11-15T03:24:53.656 INFO:scrape:5 jobs: ['5620862', '5620953', '5620687', '5620820', '5620729'] 2020-11-15T03:24:53.656 INFO:scrape:suites intersection: ['2-workunit/iozone}}', 'clusters/1-mds-1-client-coloc', 'conf/{client', 'fs/thrash/{begin', 'mds', 'mon', 'mount/fuse', 'msgr-failures/osd-mds-delay', 'osd}', 'overrides/{frag_enable', 'session_timeout', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.656 INFO:scrape:suites union: ['2-workunit/iozone}}', 'clusters/1-mds-1-client-coloc', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'fs/thrash/{begin', 'mds', 'mon', 'mount/fuse', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore-ec/filestore-xfs', 'osd}', 'overrides/{frag_enable', 'session_timeout', 'tasks/{1-thrash/mon', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.656 INFO:scrape: 2020-11-15T03:24:53.657 INFO:scrape:Failure: Test failure: test_21501 (tasks.cephfs.test_volume_client.TestVolumeClient) 2020-11-15T03:24:53.657 INFO:scrape:1 jobs: ['5620817'] 2020-11-15T03:24:53.657 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/volumes/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/volume-client}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.658 INFO:scrape: 2020-11-15T03:24:53.658 INFO:scrape:Dead: b'2020-11-15T02:34:16.706 DEBUG:teuthology.exit:Finished running handlers' 2020-11-15T03:24:53.658 INFO:scrape:ceph version 16.0.0-7160-g4090b2dd (4090b2ddb3668718593e366f72a099e049d8e78a) pacific (dev) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7fb61ca0f980] 2: gsignal() 3: abort() 4: /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8c957) [0x7fb61c4fc957] 5: /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x92ae6) [0x7fb61c502ae6] 6: /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x92b21) [0x7fb61c502b21] 7: /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x92d54) [0x7fb61c502d54] 8: /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8e837) [0x7fb61c4fe837] 9: ceph-fuse(+0x13001e) [0x556b7c73001e] 10: (Client::_create(Inode*, char const*, int, unsigned int, boost::intrusive_ptr*, Fh**, int, int, int, char const*, bool*, UserPerm const&)+0x4bb) [0x556b7c6e531b] 11: (Client::_ll_create(Inode*, char const*, unsigned int, int, boost::intrusive_ptr*, int, Fh**, UserPerm const&)+0x21d) [0x556b7c6e67ed] 12: (Client::ll_create(Inode*, char const*, unsigned int, int, stat*, Inode**, Fh**, UserPerm const&)+0x110) [0x556b7c6e7180] 13: ceph-fuse(+0x69dcb) [0x556b7c669dcb] 14: /lib/x86_64-linux-gnu/libfuse.so.2(+0x13cbd) [0x7fb625ce5cbd] 15: /lib/x86_64-linux-gnu/libfuse.so.2(+0x156c1) [0x7fb625ce76c1] 16: /lib/x86_64-linux-gnu/libfuse.so.2(+0x11e68) [0x7fb625ce3e68] 17: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fb61ca046db] 18: clone() 2020-11-15T03:24:53.659 INFO:scrape:1 jobs: ['5620868'] 2020-11-15T03:24:53.659 INFO:scrape:suites: ['clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin', 'k-testing}', 'kclient-overrides/{distro/testing/{flavor/ubuntu_latest', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-bitmap', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.659 INFO:scrape: 2020-11-15T03:24:53.660 INFO:scrape:Failure: Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_trim_caps' 2020-11-15T03:24:53.660 INFO:scrape:1 jobs: ['5620640'] 2020-11-15T03:24:53.660 INFO:scrape:suites: ['clusters/small-cluster', 'conf/{client', 'fs/bugs/client_trim_caps/{begin', 'mds', 'mon', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/trim-i22073}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.660 INFO:scrape: 2020-11-15T03:24:53.661 INFO:scrape:Failure: Command failed (workunit test libcephfs/test.sh) on smithi199 with status 134: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/libcephfs/test.sh' 2020-11-15T03:24:53.661 INFO:scrape:1 jobs: ['5620642'] 2020-11-15T03:24:53.662 INFO:scrape:suites: ['clusters/1-mds-1-client-coloc', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/libcephfs/{begin', 'mds', 'mon', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/libcephfs}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.662 INFO:scrape: 2020-11-15T03:24:53.662 INFO:scrape:Failure: Command failed (workunit test fs/test_python.sh) on smithi075 with status 134: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_python.sh' 2020-11-15T03:24:53.663 INFO:scrape:1 jobs: ['5620811'] 2020-11-15T03:24:53.663 INFO:scrape:suites: ['clusters/1-mds-1-client-coloc', 'conf/{client', 'distro/{rhel_8}', 'fs/libcephfs/{begin', 'mds', 'mon', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/libcephfs_python}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.663 INFO:scrape: 2020-11-15T03:24:53.664 INFO:scrape:Failure: Test failure: test_df_for_valid_file (tasks.cephfs.test_cephfs_shell.TestDF) 2020-11-15T03:24:53.664 INFO:scrape:1 jobs: ['5620647'] 2020-11-15T03:24:53.664 INFO:scrape:suites: ['clusters/1-mds-1-client-coloc', 'conf/{client', 'distro/{rhel_8}', 'fs/shell/{begin', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/cephfs-shell}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-15T03:24:53.664 INFO:scrape: 2020-11-15T03:24:53.684 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/ failed: 169 dead: 4 running: 0 waiting: 0 queued: 0 passed: 162 Fail ================================================================= [5620639] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} ----------------------------------------------------------------- time: 00:08:07 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620639/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620639/ sentry: https://sentry.ceph.com/organizations/ceph/?query=40ebb663f45846bcb897c304fc777a32 Command failed (workunit test suites/fsstress.sh) on smithi205 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620640] fs/bugs/client_trim_caps/{begin clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/trim-i22073} ----------------------------------------------------------------- time: 00:07:20 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620640/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620640/ sentry: https://sentry.ceph.com/organizations/ceph/?query=61baff5624ef48ce9b0cc969958dc58b Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_trim_caps' [5620642] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs} ----------------------------------------------------------------- time: 00:07:46 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620642/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620642/ sentry: https://sentry.ceph.com/organizations/ceph/?query=01de873ba3bf4aed82431da59300c8ec Command failed (workunit test libcephfs/test.sh) on smithi199 with status 134: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/libcephfs/test.sh' [5620643] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} ----------------------------------------------------------------- time: 03:21:50 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620643/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620643/ sentry: https://sentry.ceph.com/organizations/ceph/?query=46b46e7a084e4fe2b81c30eb7436b7a1 Command failed (workunit test suites/iozone.sh) on smithi073 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620644] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:08:03 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620644/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620644/ sentry: https://sentry.ceph.com/organizations/ceph/?query=37e6ae403d884b0dbada7793bd03f8e9 Test failure: test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) [5620645] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} ----------------------------------------------------------------- time: 00:29:38 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620645/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620645/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b84de90c119b485b9778883a66ff3615 Test failure: test_clients (tasks.cephfs.test_failover.TestMultiFilesystems) [5620646] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} ----------------------------------------------------------------- time: 00:10:48 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620646/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620646/ sentry: https://sentry.ceph.com/organizations/ceph/?query=264bb839edc349cfac84ce9bd9a26d8c Command failed (workunit test fs/misc/acl.sh) on smithi172 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh' [5620647] fs/shell/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cephfs-shell} ----------------------------------------------------------------- time: 00:09:28 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620647/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620647/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a76ae58a48a140a99a313dd45a149194 Test failure: test_df_for_valid_file (tasks.cephfs.test_cephfs_shell.TestDF) [5620648] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:11:24 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620648/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620648/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ab0bd7024d0b431da3f4b38b314b1f0b Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi047 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' [5620649] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:08:32 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620649/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620649/ sentry: https://sentry.ceph.com/organizations/ceph/?query=69749f9a79f143eaa6d363b20002984a Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi036 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' [5620650] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} ----------------------------------------------------------------- time: 00:07:52 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620650/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620650/ sentry: https://sentry.ceph.com/organizations/ceph/?query=002f2778c5ae4f168360b59afae1c055 Command failed (workunit test suites/blogbench.sh) on smithi078 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh' [5620652] fs/verify/{begin centos_latest clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench validater/lockdep} ----------------------------------------------------------------- time: 00:07:44 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620652/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620652/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fe1ce9ee0f3e4de8b5a4b92a63e4ce8a Command failed (workunit test suites/dbench.sh) on smithi002 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [5620653] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} ----------------------------------------------------------------- time: 00:08:45 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620653/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620653/ sentry: https://sentry.ceph.com/organizations/ceph/?query=44addd3d33544f59b94aab5ff7920eb9 Test failure: test_15303 (tasks.cephfs.test_volume_client.TestVolumeClient) [5620654] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/direct_io} ----------------------------------------------------------------- time: 00:08:33 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620654/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620654/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0394c77ad8d04da58910c2f0b4d3f061 Command failed (workunit test direct_io/big.sh) on smithi118 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh' [5620659] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:12:38 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620659/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620659/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a19185d64802425ca9d5a06cdfec276b Command failed (workunit test suites/fsstress.sh) on smithi200 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620660] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/dbench} ----------------------------------------------------------------- time: 00:08:24 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620660/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620660/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d9390fdc9ef043f58e84e5bfcd67ebf3 Command failed (workunit test suites/dbench.sh) on smithi038 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [5620662] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/alternate-pool} ----------------------------------------------------------------- time: 00:11:16 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620662/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620662/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1347086dddbe4e5c9778fa247216131a Test failure: test_rebuild_simple (tasks.cephfs.test_recovery_pool.TestRecoveryPool) [5620664] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:08:47 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620664/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620664/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fa121b3981794358bc67a99e37f3fc05 Command failed (workunit test suites/pjd.sh) on smithi107 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620667] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsx} ----------------------------------------------------------------- time: 00:08:24 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620667/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620667/ sentry: https://sentry.ceph.com/organizations/ceph/?query=84374abeff0e4b48984349cb4ebee8fc Command failed (workunit test suites/fsx.sh) on smithi032 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' [5620669] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} ----------------------------------------------------------------- time: 00:08:17 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620669/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620669/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c9021a8d27b041a9b130f066d167b2c3 Command failed (workunit test fs/misc/trivial_sync.sh) on smithi079 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/trivial_sync.sh' [5620671] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair} ----------------------------------------------------------------- time: 00:08:01 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620671/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620671/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c14839fdd5db4d6188f9a4d09cec03e2 Test failure: test_backtrace_repair (tasks.cephfs.test_auto_repair.TestMDSAutoRepair) [5620673] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iozone} ----------------------------------------------------------------- time: 03:09:45 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620673/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620673/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a9ede7fc94f1486993630a0d8a687024 Command failed (workunit test suites/iozone.sh) on smithi187 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620674] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/ffsb}} ----------------------------------------------------------------- time: 00:08:04 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620674/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620674/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c9f17c109dc04da09fef2140ddd7d2ee Command failed (workunit test suites/ffsb.sh) on smithi189 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' [5620677] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:11:28 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620677/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620677/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f4f0f15a58cc413f9b570b203d6467bf Command failed (workunit test suites/pjd.sh) on smithi174 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620679] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:08:32 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620679/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620679/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0f980cbfcae64afaa722a0fa5459a451 Command failed (workunit test suites/pjd.sh) on smithi094 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620681] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} ----------------------------------------------------------------- time: 00:07:46 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620681/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620681/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a26a34e80efa4b9fba67fba6405c1c86 Command failed (workunit test suites/dbench.sh) on smithi116 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [5620682] fs/verify/{begin centos_latest clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress validater/valgrind} ----------------------------------------------------------------- time: 00:23:19 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620682/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620682/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d6d05271c6d447c88edabef2814f30e2 Command failed (workunit test suites/fsstress.sh) on smithi040 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620683] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} ----------------------------------------------------------------- time: 00:19:28 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620683/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620683/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0bbdc6970ea94152b93d48cbaa81d2fc Test failure: test_subvolume_group_create_with_desired_data_pool_layout (tasks.cephfs.test_volumes.TestSubvolumeGroups) [5620687] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/iozone}} ----------------------------------------------------------------- time: 03:14:58 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620687/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620687/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a903b0da77d44fdc988b3228d6a664c2 Command failed (workunit test suites/iozone.sh) on smithi190 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620688] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/kernel_untar_build} ----------------------------------------------------------------- time: 00:08:24 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620688/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620688/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5b83804512d54a2e8764a982140fa4ee Command failed (workunit test kernel_untar_build.sh) on smithi089 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' [5620693] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} ----------------------------------------------------------------- time: 00:09:09 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620693/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620693/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3ed0c0617a8740e981ec2c7e26a70d87 Command failed on smithi094 with status 255: 'mpiexec -f /home/ubuntu/cephtest/mpi-hosts /home/ubuntu/cephtest/binary/usr/local/bin/ior -e -w -r -W -b 10m -a POSIX -o /home/ubuntu/cephtest/gmnt/ior.testfile' [5620694] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:08:57 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620694/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620694/ sentry: https://sentry.ceph.com/organizations/ceph/?query=29d4066fc06a494ca97381235c18753b Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi072 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' [5620696] fs/upgrade/volumes/import-legacy/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} ----------------------------------------------------------------- time: 03:38:02 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620696/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620696/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ad92c3bbf6074d678bde681f8a76cda6 Command failed (workunit test fs/upgrade/volume_client) on smithi155 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 ACTION=verify adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/upgrade/volume_client' [5620697] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/norstats} ----------------------------------------------------------------- time: 00:08:11 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620697/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620697/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b31b3b361c8d4b1380883627054dfc01 Command failed (workunit test fs/norstats/kernel_untar_tar.sh) on smithi085 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh' [5620698] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-readahead} ----------------------------------------------------------------- time: 00:10:52 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620698/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620698/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9637a5fb6e464d85bea62b5f1f5985e5 Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead) [5620700] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:11:04 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620700/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620700/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ba2540fba1054ff6aaa4ffb3e325cd64 Command failed (workunit test suites/fsstress.sh) on smithi186 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620704] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/test_o_trunc} ----------------------------------------------------------------- time: 00:07:48 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620704/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620704/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bc616cb6a28745ee9ec6e410f0241dc8 Command failed on smithi184 with status 1: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp' [5620705] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:08:29 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620705/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620705/ sentry: https://sentry.ceph.com/organizations/ceph/?query=61b7bf1b3e064ab0b22d857d37e21002 Command failed (workunit test suites/pjd.sh) on smithi018 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620710] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} ----------------------------------------------------------------- time: 00:11:16 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620710/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620710/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3f404ed9f4fb4cef90a21342f18a35c0 Command failed (workunit test suites/fsstress.sh) on smithi008 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620712] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} ----------------------------------------------------------------- time: 00:11:11 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620712/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620712/ sentry: https://sentry.ceph.com/organizations/ceph/?query=14ecd03443f04556886e86ff17a5d2b8 Command failed (workunit test fs/misc/acl.sh) on smithi097 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh' [5620714] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/cfuse_workunit_trivial_sync}} ----------------------------------------------------------------- time: 00:08:16 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620714/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620714/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d42786a2a57b4bbdad5a0f8a2499d1da Command failed (workunit test fs/misc/trivial_sync.sh) on smithi066 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/trivial_sync.sh' [5620715] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} ----------------------------------------------------------------- time: 00:08:00 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620715/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620715/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8a3113032d2d47c98c55619d7ebfa4f8 Command failed (workunit test suites/ffsb.sh) on smithi192 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' [5620716] fs/verify/{begin centos_latest clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench validater/lockdep} ----------------------------------------------------------------- time: 00:07:51 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620716/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620716/ sentry: https://sentry.ceph.com/organizations/ceph/?query=78ec5d9b7f924bb78bbf8b6ff67112d5 Command failed (workunit test suites/dbench.sh) on smithi100 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [5620717] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} ----------------------------------------------------------------- time: 00:14:13 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620717/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620717/ sentry: https://sentry.ceph.com/organizations/ceph/?query=24c6b238a9bc4a65b1a6b434d8896fa1 Test failure: test_clone_subvolume_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) [5620718] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/direct_io} ----------------------------------------------------------------- time: 00:08:31 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620718/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620718/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bc69a7d541904576b2a84470cf3008e7 Command failed (workunit test direct_io/big.sh) on smithi092 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh' [5620723] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub} ----------------------------------------------------------------- time: 00:10:56 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620723/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620723/ sentry: https://sentry.ceph.com/organizations/ceph/?query=48d392210724443faea9905e9625702f Test failure: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) [5620724] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/ffsb}} ----------------------------------------------------------------- time: 00:08:23 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620724/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620724/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5ce8a951ffc941089feb062d34963f23 Command failed (workunit test suites/ffsb.sh) on smithi131 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' [5620725] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/dbench} ----------------------------------------------------------------- time: 00:08:30 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620725/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620725/ sentry: https://sentry.ceph.com/organizations/ceph/?query=df0b8155e8494caaa8799307880b6dcd Command failed (workunit test suites/dbench.sh) on smithi063 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [5620729] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/iozone}} ----------------------------------------------------------------- time: 03:12:56 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620729/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620729/ sentry: https://sentry.ceph.com/organizations/ceph/?query=562408e41cd44bc89798545fda079a6d Command failed (workunit test suites/iozone.sh) on smithi166 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620731] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsx} ----------------------------------------------------------------- time: 00:11:52 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620731/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620731/ sentry: https://sentry.ceph.com/organizations/ceph/?query=02a40d39ca604a9db7bf1fb7eee4bf5b Command failed (workunit test suites/fsx.sh) on smithi124 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' [5620734] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} ----------------------------------------------------------------- time: 03:21:53 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620734/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620734/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e7cc230c942e4782bb316f19b80bec25 Command failed (workunit test suites/iozone.sh) on smithi018 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620735] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} ----------------------------------------------------------------- time: 00:08:30 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620735/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620735/ sentry: https://sentry.ceph.com/organizations/ceph/?query=48684462532948e7ac31efd40cca3134 Command failed on smithi159 with status 1: 'mpiexec -f /home/ubuntu/cephtest/mpi-hosts /home/ubuntu/cephtest/mdtest-1.9.3/mdtest -d /home/ubuntu/cephtest/gmnt -I 20 -z 5 -b 2 -R' [5620736] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:08:11 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620736/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620736/ sentry: https://sentry.ceph.com/organizations/ceph/?query=479f3d0db80a45d59235e428fcb2c62c Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi079 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' [5620740] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} ----------------------------------------------------------------- time: 00:13:11 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620740/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620740/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a002f63531e9463a8c45a19a70d45d83 Command failed (workunit test suites/fsstress.sh) on smithi043 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620741] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iozone} ----------------------------------------------------------------- time: 03:12:40 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620741/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620741/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e6e94f19d9a84ddf941b2b9bbfdfedf7 Command failed (workunit test suites/iozone.sh) on smithi143 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620742] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/cfuse_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:11:35 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620742/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620742/ sentry: https://sentry.ceph.com/organizations/ceph/?query=17300f121fa647edb0bf6e2d8dc28b50 Command failed (workunit test suites/fsstress.sh) on smithi186 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620744] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-full} ----------------------------------------------------------------- time: 00:07:22 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620744/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620744/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c5b8aff67e974c0d8c8a2de471494aad Test failure: test_barrier (tasks.cephfs.test_full.TestClusterFull) [5620745] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:10:59 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620745/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620745/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a365b10f4806487f81db781c451499ec Command failed (workunit test suites/pjd.sh) on smithi097 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620747] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:11:07 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620747/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620747/ sentry: https://sentry.ceph.com/organizations/ceph/?query=33dbe21bd79147f9a99be2f90066f711 Command failed (workunit test suites/pjd.sh) on smithi200 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620749] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} ----------------------------------------------------------------- time: 00:11:42 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620749/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620749/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6e496657454746b286d120240e6f5b0f Command failed (workunit test suites/fsstress.sh) on smithi106 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620750] fs/verify/{begin centos_latest clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress validater/valgrind} ----------------------------------------------------------------- time: 00:21:55 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620750/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620750/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8420dd26ab6449d1857cb5952269d28e Command failed (workunit test suites/fsstress.sh) on smithi089 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620751] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} ----------------------------------------------------------------- time: 00:13:09 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620751/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620751/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b3744984523f42889e570a2ab11f41d6 Test failure: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) [5620754] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:11:25 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620754/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620754/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2e9bf73660f242e6bda8d6215d6ddaa7 Command failed (workunit test suites/pjd.sh) on smithi131 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620755] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/kernel_untar_build} ----------------------------------------------------------------- time: 00:11:23 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620755/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620755/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9e4e688bdacf4c558e2fa3f7fa6fce10 Command failed (workunit test kernel_untar_build.sh) on smithi008 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' [5620760] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} ----------------------------------------------------------------- time: 00:11:20 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620760/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620760/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e3b96291756644be974e47f565c4cb4b Command failed (workunit test fs/misc/trivial_sync.sh) on smithi002 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/trivial_sync.sh' [5620762] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/norstats} ----------------------------------------------------------------- time: 00:11:41 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620762/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620762/ sentry: https://sentry.ceph.com/organizations/ceph/?query=645baab902424298b42640f12867a026 Command failed (workunit test fs/norstats/kernel_untar_tar.sh) on smithi026 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh' [5620765] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/ffsb}} ----------------------------------------------------------------- time: 00:08:33 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620765/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620765/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bed936093895465098d1ec2fc674552c Command failed (workunit test suites/ffsb.sh) on smithi186 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' [5620768] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/quota} ----------------------------------------------------------------- time: 00:08:18 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620768/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620768/ sentry: https://sentry.ceph.com/organizations/ceph/?query=54acd7d840504a33a86217ce7e284bf7 Test failure: test_remote_update_write (tasks.cephfs.test_quota.TestQuota) [5620769] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/test_o_trunc} ----------------------------------------------------------------- time: 00:08:24 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620769/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620769/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ce203cc9bb1d464a9088cdd04aa4e438 Command failed on smithi170 with status 1: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp' [5620770] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/iozone}} ----------------------------------------------------------------- time: 03:09:10 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620770/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620770/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e886846fbc7844f59b597530dd421261 Command failed (workunit test suites/iozone.sh) on smithi097 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620775] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/filestore-xfs overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} ----------------------------------------------------------------- time: 00:07:20 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620775/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620775/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e95f9c82cc0949269bd25f2f8919a878 Command failed (workunit test suites/fsstress.sh) on smithi169 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620777] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:09:03 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620777/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620777/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ed4104920d9b4c00ad94368535226127 Test failure: test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) [5620778] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} ----------------------------------------------------------------- time: 00:28:37 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620778/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620778/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8b90f9b419a7405083caa64908669c2e Test failure: test_clients (tasks.cephfs.test_failover.TestMultiFilesystems) [5620779] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} ----------------------------------------------------------------- time: 00:07:48 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620779/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620779/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0f79ce18706144149ee81348053af9cc Command failed (workunit test fs/misc/acl.sh) on smithi200 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh' [5620780] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:09:06 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620780/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620780/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f81f1dd916204534977b6c51929727c8 Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi168 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' [5620781] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:08:44 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620781/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620781/ sentry: https://sentry.ceph.com/organizations/ceph/?query=95f8b7979a2e4709837667600c34fbfd Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi016 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' [5620782] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} ----------------------------------------------------------------- time: 00:07:34 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620782/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620782/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a139ad39886a40ee8609d4de6b35bde1 Command failed (workunit test suites/blogbench.sh) on smithi160 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh' [5620783] fs/verify/{begin centos_latest clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench validater/lockdep} ----------------------------------------------------------------- time: 00:07:27 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620783/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620783/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b54ba426a5f14ea993f9c1185c1d34ce Command failed (workunit test suites/dbench.sh) on smithi002 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [5620784] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} ----------------------------------------------------------------- time: 00:12:07 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620784/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620784/ sentry: https://sentry.ceph.com/organizations/ceph/?query=164f01d4682c46059aa2a643e24eef18 Test failure: test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) [5620785] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/direct_io} ----------------------------------------------------------------- time: 00:08:29 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620785/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620785/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8f5f077dedc64d08b3d8c4eb0bbded49 Command failed (workunit test direct_io/big.sh) on smithi052 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh' [5620790] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:08:14 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620790/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620790/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ed965b4a47e94d7a94ef3272df3f6a94 Command failed (workunit test suites/fsstress.sh) on smithi070 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620793] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/dbench} ----------------------------------------------------------------- time: 00:08:32 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620793/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620793/ sentry: https://sentry.ceph.com/organizations/ceph/?query=36f2f329aafb4396b609ebff4a68c7cb Command failed (workunit test suites/dbench.sh) on smithi026 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [5620796] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/test_journal_migration} ----------------------------------------------------------------- time: 00:07:38 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620796/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620796/ sentry: https://sentry.ceph.com/organizations/ceph/?query=49bca73410044240b94fb82d0f7c52cb Test failure: test_journal_migration (tasks.cephfs.test_journal_migration.TestJournalMigration) [5620797] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:11:51 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620797/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620797/ sentry: https://sentry.ceph.com/organizations/ceph/?query=69b0ba6372b346c989eac6d88686360f Command failed (workunit test suites/pjd.sh) on smithi078 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620799] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsx} ----------------------------------------------------------------- time: 00:07:53 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620799/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620799/ sentry: https://sentry.ceph.com/organizations/ceph/?query=673481d9a4d240859ddea03eab794c46 Command failed (workunit test suites/fsx.sh) on smithi038 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' [5620802] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} ----------------------------------------------------------------- time: 00:08:15 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620802/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620802/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d8529f45870049e59efc20d3b9b40a52 Command failed (workunit test fs/misc/trivial_sync.sh) on smithi122 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/trivial_sync.sh' [5620806] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iozone} ----------------------------------------------------------------- time: 03:10:02 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620806/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620806/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8a61a715ae20470f90e7f3b435c93e16 Command failed (workunit test suites/iozone.sh) on smithi047 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620807] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/ffsb}} ----------------------------------------------------------------- time: 00:08:22 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620807/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620807/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d065dd39f8d84c1d863ad4ac28eae39e Command failed (workunit test suites/ffsb.sh) on smithi040 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' [5620809] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:08:17 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620809/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620809/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e1a7746d8b764838b9d163fabbf9fce8 Command failed (workunit test suites/pjd.sh) on smithi014 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620811] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs_python} ----------------------------------------------------------------- time: 00:10:48 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620811/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620811/ sentry: https://sentry.ceph.com/organizations/ceph/?query=885015f200d84d379f801cf425a500b7 Command failed (workunit test fs/test_python.sh) on smithi075 with status 134: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_python.sh' [5620813] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:08:19 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620813/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620813/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1f340b90bc37411da40479e02c89b2f2 Command failed (workunit test suites/pjd.sh) on smithi042 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620815] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} ----------------------------------------------------------------- time: 00:11:30 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620815/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620815/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3b83cc605e174f0eac174a7627a0ebc8 Command failed (workunit test suites/dbench.sh) on smithi198 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [5620816] fs/verify/{begin centos_latest clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress validater/valgrind} ----------------------------------------------------------------- time: 00:23:02 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620816/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620816/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3d0a410770694833a2a4e7ebb5c2bef1 Command failed (workunit test suites/fsstress.sh) on smithi033 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620817] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} ----------------------------------------------------------------- time: 00:11:50 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620817/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620817/ sentry: https://sentry.ceph.com/organizations/ceph/?query=89f1007328fd4a38b59c75384630531f Test failure: test_21501 (tasks.cephfs.test_volume_client.TestVolumeClient) [5620820] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/iozone}} ----------------------------------------------------------------- time: 03:12:49 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620820/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620820/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4ff30eb15a97431a8eb7a5d0a8ec3d3a Command failed (workunit test suites/iozone.sh) on smithi038 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620821] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/kernel_untar_build} ----------------------------------------------------------------- time: 00:11:52 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620821/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620821/ sentry: https://sentry.ceph.com/organizations/ceph/?query=95f6850e49d64c5cb0f56ea1289a996c Command failed (workunit test kernel_untar_build.sh) on smithi159 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' [5620822] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/admin} ----------------------------------------------------------------- time: 00:16:21 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620822/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620822/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9da70f36c97143e4954f2db8a99bd334 Test failure: test_client_config (tasks.cephfs.test_admin.TestConfigCommands) [5620825] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} ----------------------------------------------------------------- time: 03:17:08 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620825/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620825/ sentry: https://sentry.ceph.com/organizations/ceph/?query=20fecc41f91e4609b65084bf7a1c59ae Command failed (workunit test suites/iozone.sh) on smithi151 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620826] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} ----------------------------------------------------------------- time: 00:09:00 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620826/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620826/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f564cac7adb64ac2b2b1d09866f7ea9f Command failed on smithi089 with status 255: 'mpiexec -f /home/ubuntu/cephtest/mpi-hosts /home/ubuntu/cephtest/binary/usr/local/bin/ior -e -w -r -W -b 10m -a POSIX -o /home/ubuntu/cephtest/gmnt/ior.testfile' [5620827] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:11:12 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620827/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620827/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ce685958b70f49a7907633a0d6f0554b Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi169 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' [5620830] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/norstats} ----------------------------------------------------------------- time: 00:08:22 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620830/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620830/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1f0cd34a1cca453d95b120385dc3f538 Command failed (workunit test fs/norstats/kernel_untar_tar.sh) on smithi116 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh' [5620833] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:07:57 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620833/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620833/ sentry: https://sentry.ceph.com/organizations/ceph/?query=328833e793194530a2c2370c4f475ce2 Command failed (workunit test suites/fsstress.sh) on smithi116 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620837] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/test_o_trunc} ----------------------------------------------------------------- time: 00:11:29 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620837/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620837/ sentry: https://sentry.ceph.com/organizations/ceph/?query=44aede58c14e44f1ad6fce8557e5defc Command failed on smithi144 with status 1: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp' [5620839] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:11:09 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620839/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620839/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cff998ca007f4623876a4944a0d3ea71 Command failed (workunit test suites/pjd.sh) on smithi174 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620842] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/backtrace} ----------------------------------------------------------------- time: 00:08:26 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620842/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620842/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dc10cd9476d641a6aa7399b07394d39f Test failure: test_backtrace (tasks.cephfs.test_backtrace.TestBacktrace) [5620844] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} ----------------------------------------------------------------- time: 00:07:51 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620844/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620844/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c3e09d653cd347cbada9d6f021a77899 Command failed (workunit test suites/fsstress.sh) on smithi121 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620846] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} ----------------------------------------------------------------- time: 00:08:10 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620846/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620846/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e1632322370d4c9c8bc9eee6d40bfd70 Command failed (workunit test fs/misc/acl.sh) on smithi201 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh' [5620848] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/cfuse_workunit_trivial_sync}} ----------------------------------------------------------------- time: 00:08:26 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620848/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620848/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f91b1b7f3cab42e79cc46b097334cdee Command failed (workunit test fs/misc/trivial_sync.sh) on smithi169 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/trivial_sync.sh' [5620849] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} ----------------------------------------------------------------- time: 00:11:15 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620849/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620849/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8e73e4b40ee841f99c1c68606f1c2a97 Command failed (workunit test suites/ffsb.sh) on smithi089 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' [5620850] fs/verify/{begin centos_latest clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench validater/lockdep} ----------------------------------------------------------------- time: 00:08:12 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620850/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620850/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fb3c238629fa4afdbbd13688ccec71d5 Command failed (workunit test suites/dbench.sh) on smithi039 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [5620851] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} ----------------------------------------------------------------- time: 00:20:13 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620851/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620851/ sentry: https://sentry.ceph.com/organizations/ceph/?query=12fed09842e74752ae85aaa73b5b0c18 Test failure: test_subvolume_group_create_with_desired_data_pool_layout (tasks.cephfs.test_volumes.TestSubvolumeGroups) [5620852] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/direct_io} ----------------------------------------------------------------- time: 00:07:50 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620852/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620852/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f387385b98be469d859a8483e57f02ae Command failed (workunit test direct_io/big.sh) on smithi032 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh' [5620857] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/ffsb}} ----------------------------------------------------------------- time: 00:08:03 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620857/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620857/ sentry: https://sentry.ceph.com/organizations/ceph/?query=800bf41afa164e3d8e802bbdff00feeb Command failed (workunit test suites/ffsb.sh) on smithi025 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' [5620858] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/dbench} ----------------------------------------------------------------- time: 00:11:58 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620858/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620858/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a58ebdd08a094d77a216631fddc40190 Command failed (workunit test suites/dbench.sh) on smithi055 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [5620862] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/iozone}} ----------------------------------------------------------------- time: 03:11:34 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620862/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620862/ sentry: https://sentry.ceph.com/organizations/ceph/?query=eff37c0a01ba41ea9a31554ec2ad52f8 Command failed (workunit test suites/iozone.sh) on smithi096 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620865] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsx} ----------------------------------------------------------------- time: 00:12:09 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620865/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620865/ sentry: https://sentry.ceph.com/organizations/ceph/?query=138c44fcb9f349cab026b82b0eae4f5c Command failed (workunit test suites/fsx.sh) on smithi052 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' [5620867] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery} ----------------------------------------------------------------- time: 00:14:05 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620867/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620867/ sentry: https://sentry.ceph.com/organizations/ceph/?query=532338fe85b54f4397feb9bc9056588d Test failure: test_basic (tasks.cephfs.test_client_recovery.TestClientRecovery) [5620869] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} ----------------------------------------------------------------- time: 00:08:56 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620869/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620869/ sentry: https://sentry.ceph.com/organizations/ceph/?query=42756ac36d71470583357d96c4502d84 Command failed on smithi196 with status 1: 'mpiexec -f /home/ubuntu/cephtest/mpi-hosts /home/ubuntu/cephtest/mdtest-1.9.3/mdtest -d /home/ubuntu/cephtest/gmnt -I 20 -z 5 -b 2 -R' [5620870] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:09:39 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620870/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620870/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0a407b26b5b540f4a9467cf8372e66cd Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi067 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' [5620874] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iozone} ----------------------------------------------------------------- time: 03:10:16 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620874/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620874/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dd72cf12b08c4d8fa6e7089ec63151ca Command failed (workunit test suites/iozone.sh) on smithi169 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620875] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/cfuse_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:09:12 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620875/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620875/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9f37b42d4d3540e1a3875e6809349160 Command failed (workunit test suites/fsstress.sh) on smithi066 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620877] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:11:09 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620877/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620877/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c1b7aa32a9a44ad190927c12c5553389 Command failed (workunit test suites/pjd.sh) on smithi196 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620879] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:07:53 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620879/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620879/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b160ba5ead6d498aa09ebdbef067d58d Command failed (workunit test suites/pjd.sh) on smithi171 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620881] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} ----------------------------------------------------------------- time: 00:11:27 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620881/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620881/ sentry: https://sentry.ceph.com/organizations/ceph/?query=016e9588a317407e8b876e27c711eca0 Command failed (workunit test suites/fsstress.sh) on smithi047 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620882] fs/verify/{begin centos_latest clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress validater/valgrind} ----------------------------------------------------------------- time: 00:21:57 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620882/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620882/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d306f8ba7efa43c5acf4c2c4e314b6ad Command failed (workunit test suites/fsstress.sh) on smithi085 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620883] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} ----------------------------------------------------------------- time: 00:13:27 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620883/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620883/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dd051a2223274d89b25a0d2dfe5fd4aa Test failure: test_clone_subvolume_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) [5620887] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:08:23 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620887/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620887/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9608b58878ae4fb3ac148dbd1bb0ad76 Command failed (workunit test suites/pjd.sh) on smithi006 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620888] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} ----------------------------------------------------------------- time: 00:14:02 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620888/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620888/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f7047c095169443fa0de11eed9261e91 Command failed (workunit test suites/fsstress.sh) on smithi097 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620889] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/kernel_untar_build} ----------------------------------------------------------------- time: 00:11:26 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620889/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620889/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0e8e4425eddc47deac3216d76f432a14 Command failed (workunit test kernel_untar_build.sh) on smithi084 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' [5620893] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} ----------------------------------------------------------------- time: 00:08:31 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620893/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620893/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0c8ac5c0c3ae4ec78340a18e364c87d0 Command failed (workunit test fs/misc/trivial_sync.sh) on smithi055 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/trivial_sync.sh' [5620895] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/fragment} ----------------------------------------------------------------- time: 00:11:07 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620895/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620895/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3365d0537d6c45f1b1c254730c48955c Test failure: test_deep_split (tasks.cephfs.test_fragment.TestFragmentation) [5620896] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/norstats} ----------------------------------------------------------------- time: 00:08:08 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620896/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620896/ sentry: https://sentry.ceph.com/organizations/ceph/?query=586c161c35ee4a7e911cedb154b454e3 Command failed (workunit test fs/norstats/kernel_untar_tar.sh) on smithi060 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh' [5620898] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/ffsb}} ----------------------------------------------------------------- time: 00:08:42 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620898/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620898/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f479c255b3344fb0bded4bb1cef62791 Command failed (workunit test suites/ffsb.sh) on smithi150 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' [5620902] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/test_o_trunc} ----------------------------------------------------------------- time: 00:11:50 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620902/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620902/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9eab2c9d7cf04e7dac9f4a0a46bf16b6 Command failed on smithi042 with status 1: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp' [5620904] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/iozone}} ----------------------------------------------------------------- time: 03:09:30 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620904/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620904/ sentry: https://sentry.ceph.com/organizations/ceph/?query=02b66db443664057a5fa83d784bfc969 Command failed (workunit test suites/iozone.sh) on smithi186 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620908] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-full} ----------------------------------------------------------------- time: 00:15:36 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620908/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620908/ sentry: https://sentry.ceph.com/organizations/ceph/?query=143f80d012db48bc9977629689b29969 Test failure: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) [5620909] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} ----------------------------------------------------------------- time: 00:07:49 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620909/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620909/ sentry: https://sentry.ceph.com/organizations/ceph/?query=14e28077e5f54ddb8ce59fd576cb493c Command failed (workunit test suites/fsstress.sh) on smithi193 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620910] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} ----------------------------------------------------------------- time: 03:33:48 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620910/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620910/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b2b39f6ae2bc436f8ef8fbb5e2d8af84 Command failed (workunit test suites/iozone.sh) on smithi066 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620911] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:08:30 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620911/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620911/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bb5cdf4d1af74b899dc849c474db30cf Test failure: test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) [5620912] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} ----------------------------------------------------------------- time: 00:28:13 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620912/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620912/ sentry: https://sentry.ceph.com/organizations/ceph/?query=31eadd13f42246368faabf99dc3665c1 Test failure: test_clients (tasks.cephfs.test_failover.TestMultiFilesystems) [5620913] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} ----------------------------------------------------------------- time: 00:08:10 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620913/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620913/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a6cab8cfc4594c4d95e1765fdba93c11 Command failed (workunit test fs/misc/acl.sh) on smithi198 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh' [5620914] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:08:23 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620914/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620914/ sentry: https://sentry.ceph.com/organizations/ceph/?query=54609915c4754f7c840e39da06c6cfde Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi059 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' [5620915] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:08:13 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620915/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620915/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0d773ee6c69347c488b6000b0b529618 Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi117 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' [5620916] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} ----------------------------------------------------------------- time: 00:08:15 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620916/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620916/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f00438b5195e4a6ba9d94df2e41e5225 Command failed (workunit test suites/blogbench.sh) on smithi123 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh' [5620917] fs/verify/{begin centos_latest clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench validater/lockdep} ----------------------------------------------------------------- time: 00:08:01 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620917/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620917/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e1493b1257684a1899cdd8457a9a4431 Command failed (workunit test suites/dbench.sh) on smithi055 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [5620918] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} ----------------------------------------------------------------- time: 00:11:43 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620918/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620918/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a2d82f68545f4e328ff9029bef6924f9 Test failure: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) [5620919] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/direct_io} ----------------------------------------------------------------- time: 00:08:28 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620919/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620919/ sentry: https://sentry.ceph.com/organizations/ceph/?query=697c3ae8b5104c8c9ab43efdf2578cd0 Command failed (workunit test direct_io/big.sh) on smithi162 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh' [5620924] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:07:51 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620924/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620924/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f71d12cd645a4e2aad6c87d0e171ba68 Command failed (workunit test suites/fsstress.sh) on smithi092 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620925] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/dbench} ----------------------------------------------------------------- time: 00:08:59 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620925/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620925/ sentry: https://sentry.ceph.com/organizations/ceph/?query=198e91c3a8df4be09f0ce98e9fa38b73 Command failed (workunit test suites/dbench.sh) on smithi151 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [5620929] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:08:40 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620929/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620929/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a14a00b251b249658b785044550b4e68 Command failed (workunit test suites/pjd.sh) on smithi179 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620933] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsx} ----------------------------------------------------------------- time: 00:08:18 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620933/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620933/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fb3ef2d3d28647bfb0182aa3f9fe10fe Command failed (workunit test suites/fsx.sh) on smithi164 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' [5620935] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} ----------------------------------------------------------------- time: 00:08:01 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620935/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620935/ sentry: https://sentry.ceph.com/organizations/ceph/?query=831779a2b2c64450bb4e6eac7f3388b1 Command failed (workunit test fs/misc/trivial_sync.sh) on smithi032 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/trivial_sync.sh' [5620939] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iozone} ----------------------------------------------------------------- time: 03:12:47 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620939/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620939/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6b6029f3344647c0baeddebad9431983 Command failed (workunit test suites/iozone.sh) on smithi085 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620940] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/scrub} ----------------------------------------------------------------- time: 00:11:21 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620940/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620940/ sentry: https://sentry.ceph.com/organizations/ceph/?query=88208b714ccf4e5e8ac0ea9887248b41 Test failure: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) [5620941] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/ffsb}} ----------------------------------------------------------------- time: 00:08:43 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620941/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620941/ sentry: https://sentry.ceph.com/organizations/ceph/?query=20bd07f1754542faaf9955b9dc8d5fff Command failed (workunit test suites/ffsb.sh) on smithi055 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' [5620943] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/filestore-xfs overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:07:25 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620943/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620943/ sentry: https://sentry.ceph.com/organizations/ceph/?query=610a26f353ed46729bb9cce9a3fa8cb0 Command failed (workunit test suites/pjd.sh) on smithi150 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620945] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:11:01 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620945/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620945/ sentry: https://sentry.ceph.com/organizations/ceph/?query=210682ab958441e6a011a57c2063f452 Command failed (workunit test suites/pjd.sh) on smithi008 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [5620947] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} ----------------------------------------------------------------- time: 00:10:56 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620947/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620947/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5067f367342f4b27bb9777d7c7538249 Command failed (workunit test suites/dbench.sh) on smithi139 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [5620948] fs/verify/{begin centos_latest clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress validater/valgrind} ----------------------------------------------------------------- time: 00:21:31 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620948/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620948/ sentry: https://sentry.ceph.com/organizations/ceph/?query=036dd313cfef4a2dbe14e1ff18a5dda2 Command failed (workunit test suites/fsstress.sh) on smithi119 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620949] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} ----------------------------------------------------------------- time: 00:15:06 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620949/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620949/ sentry: https://sentry.ceph.com/organizations/ceph/?query=03340bf05a624c48b21f1e0919d96007 Test failure: test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) [5620953] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/iozone}} ----------------------------------------------------------------- time: 03:12:21 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620953/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620953/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ee779a4998984f93a6d65189f900d249 Command failed (workunit test suites/iozone.sh) on smithi047 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [5620954] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/kernel_untar_build} ----------------------------------------------------------------- time: 00:08:36 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620954/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620954/ sentry: https://sentry.ceph.com/organizations/ceph/?query=76626e6c8e56478c84098f4918bdb7ba Command failed (workunit test kernel_untar_build.sh) on smithi079 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' [5620959] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} ----------------------------------------------------------------- time: 00:09:11 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620959/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620959/ sentry: https://sentry.ceph.com/organizations/ceph/?query=50050ec929954370a30f1e18e64d9a3c Command failed on smithi086 with status 255: 'mpiexec -f /home/ubuntu/cephtest/mpi-hosts /home/ubuntu/cephtest/binary/usr/local/bin/ior -e -w -r -W -b 10m -a POSIX -o /home/ubuntu/cephtest/gmnt/ior.testfile' [5620960] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:08:50 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620960/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620960/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e39fa9b4b7d0437498f918cf86d27e31 Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi031 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' [5620962] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/norstats} ----------------------------------------------------------------- time: 00:11:40 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620962/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620962/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f71d515ab63148c19035f6a97adcbd60 Command failed (workunit test fs/norstats/kernel_untar_tar.sh) on smithi103 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh' [5620965] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:07:55 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620965/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620965/ sentry: https://sentry.ceph.com/organizations/ceph/?query=62deb17b60ef4f559a2d3473ef02d5fb Command failed (workunit test suites/fsstress.sh) on smithi036 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5620967] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/truncate_delay} ----------------------------------------------------------------- time: 00:08:27 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620967/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620967/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6fc9ec4b2a924223b8b76228c0cb405f Command failed on smithi089 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'cd $TESTDIR/mnt.0 && dd if=/dev/zero of=./foo count=100'" [5620969] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/test_o_trunc} ----------------------------------------------------------------- time: 00:08:25 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620969/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620969/ sentry: https://sentry.ceph.com/organizations/ceph/?query=df1d6fa93e1e4ac39ec4a4444ad8472a Command failed on smithi122 with status 1: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp' [5620970] fs/thrash/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:08:07 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620970/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620970/ sentry: https://sentry.ceph.com/organizations/ceph/?query=eed7ff239fcd489ba50c9168b6017ca1 Command failed (workunit test suites/pjd.sh) on smithi152 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4090b2ddb3668718593e366f72a099e049d8e78a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' Dead ================================================================= [5620692] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620692/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620692/ [5620776] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620776/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620776/ [5620868] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620868/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620868/ [5620958] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620958/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620958/ Pass ================================================================= [5620641] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls} time: 00:09:00 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620641/ [5620651] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}} time: 00:31:53 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620651/ [5620655] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/misc} time: 00:38:23 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620655/ [5620656] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/kernel_untar_build} time: 00:46:46 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620656/ [5620657] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/admin} time: 00:25:02 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620657/ [5620658] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/blogbench} time: 00:19:17 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620658/ [5620661] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/ffsb} time: 00:32:40 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620661/ [5620663] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/norstats} time: 00:13:23 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620663/ [5620665] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsstress} time: 00:13:55 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620665/ [5620666] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/asok_dump_tree} time: 00:20:29 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620666/ [5620668] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsync-tester} time: 00:12:33 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620668/ [5620670] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/test_o_trunc} time: 00:21:50 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620670/ [5620672] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iogen} time: 00:48:52 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620672/ [5620675] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/backtrace} time: 00:12:10 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620675/ [5620676] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/pjd} time: 00:13:13 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620676/ [5620678] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:21:24 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620678/ [5620680] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:04:09 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620680/ [5620684] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/direct_io} time: 00:11:06 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620684/ [5620685] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/misc} time: 00:30:01 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620685/ [5620686] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cap-flush} time: 00:11:02 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620686/ [5620689] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/blogbench} time: 00:23:12 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620689/ [5620690] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/dbench} time: 01:02:20 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620690/ [5620691] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-limits} time: 00:17:36 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620691/ [5620695] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/ffsb} time: 00:23:33 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620695/ [5620699] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsstress} time: 00:16:43 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620699/ [5620701] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsx} time: 00:25:41 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620701/ [5620702] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsync-tester} time: 00:11:20 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620702/ [5620703] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery} time: 00:29:39 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620703/ [5620706] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iogen} time: 00:55:57 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620706/ [5620707] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/damage} time: 00:23:18 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620707/ [5620708] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iozone} time: 00:17:07 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620708/ [5620709] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/pjd} time: 00:12:11 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620709/ [5620711] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:51:12 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620711/ [5620713] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:48:29 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620713/ [5620719] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/data-scan} time: 00:25:06 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620719/ [5620720] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/misc} time: 00:27:22 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620720/ [5620721] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/kernel_untar_build} time: 00:47:49 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620721/ [5620722] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/blogbench} time: 00:18:43 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620722/ [5620726] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/ffsb} time: 01:01:32 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620726/ [5620727] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/fragment} time: 00:17:37 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620727/ [5620728] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/norstats} time: 00:14:58 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620728/ [5620730] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsstress} time: 00:14:01 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620730/ [5620732] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/journal-repair} time: 00:18:59 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620732/ [5620733] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsync-tester} time: 00:13:14 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620733/ [5620737] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/test_o_trunc} time: 00:21:09 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620737/ [5620738] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush} time: 00:10:08 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620738/ [5620739] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iogen} time: 00:20:29 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620739/ [5620743] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/pjd} time: 00:12:40 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620743/ [5620746] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:20:05 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620746/ [5620748] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:01:34 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620748/ [5620752] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/direct_io} time: 00:11:28 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620752/ [5620753] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/misc} time: 00:37:30 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620753/ [5620756] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds_creation_retry} time: 00:10:55 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620756/ [5620757] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/blogbench} time: 00:24:05 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620757/ [5620758] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/dbench} time: 01:05:16 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620758/ [5620759] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/openfiletable} time: 00:12:45 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620759/ [5620761] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/ffsb} time: 00:38:23 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620761/ [5620763] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsstress} time: 00:14:33 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620763/ [5620764] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm} time: 00:10:27 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620764/ [5620766] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsx} time: 00:22:31 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620766/ [5620767] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsync-tester} time: 00:11:24 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620767/ [5620771] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iogen} time: 01:17:15 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620771/ [5620772] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iozone} time: 00:18:01 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620772/ [5620773] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/scrub} time: 00:18:19 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620773/ [5620774] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/pjd} time: 00:12:02 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620774/ [5620786] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/misc} time: 00:31:20 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620786/ [5620787] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap} time: 00:14:30 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620787/ [5620788] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/kernel_untar_build} time: 00:58:46 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620788/ [5620789] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/blogbench} time: 00:16:58 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620789/ [5620791] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/strays} time: 00:23:41 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620791/ [5620792] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:20:31 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620792/ [5620794] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/ffsb} time: 00:55:27 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620794/ [5620795] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/norstats} time: 00:13:22 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620795/ [5620798] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsstress} time: 00:14:23 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620798/ [5620800] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/truncate_delay} time: 00:11:11 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620800/ [5620801] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsync-tester} time: 00:12:25 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620801/ [5620803] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/test_o_trunc} time: 00:20:56 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620803/ [5620804] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iogen} time: 00:35:52 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620804/ [5620805] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/quota} time: 00:12:23 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620805/ [5620808] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/pjd} time: 00:12:32 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620808/ [5620810] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls} time: 00:12:45 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620810/ [5620812] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:22:03 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620812/ [5620814] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:54:44 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620814/ [5620818] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/direct_io} time: 00:11:53 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620818/ [5620819] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/misc} time: 00:38:00 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620819/ [5620823] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/blogbench} time: 00:26:19 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620823/ [5620824] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/dbench} time: 00:41:05 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620824/ [5620828] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/ffsb} time: 00:27:15 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620828/ [5620829] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/alternate-pool} time: 00:11:42 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620829/ [5620831] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsstress} time: 00:15:45 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620831/ [5620832] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/asok_dump_tree} time: 00:24:40 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620832/ [5620834] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsx} time: 00:31:32 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620834/ [5620835] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsync-tester} time: 00:10:31 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620835/ [5620836] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}} time: 00:32:14 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620836/ [5620838] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair} time: 00:11:16 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620838/ [5620840] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iogen} time: 01:03:04 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620840/ [5620841] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iozone} time: 00:15:15 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620841/ [5620843] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/pjd} time: 00:12:07 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620843/ [5620845] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:51:21 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620845/ [5620847] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:09:07 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620847/ [5620853] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/misc} time: 00:31:25 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620853/ [5620854] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cap-flush} time: 00:11:19 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620854/ [5620855] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/kernel_untar_build} time: 01:00:29 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620855/ [5620856] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/blogbench} time: 00:22:21 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620856/ [5620859] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-limits} time: 00:21:15 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620859/ [5620860] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/ffsb} time: 00:25:11 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620860/ [5620861] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/norstats} time: 00:13:57 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620861/ [5620863] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-readahead} time: 00:09:58 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620863/ [5620864] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsstress} time: 00:15:17 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620864/ [5620866] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsync-tester} time: 00:12:46 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620866/ [5620871] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/test_o_trunc} time: 00:20:56 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620871/ [5620872] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iogen} time: 00:41:38 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620872/ [5620873] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/damage} time: 00:19:38 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620873/ [5620876] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/pjd} time: 00:13:05 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620876/ [5620878] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:20:40 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620878/ [5620880] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:49:06 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620880/ [5620884] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/direct_io} time: 00:12:39 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620884/ [5620885] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/data-scan} time: 00:29:03 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620885/ [5620886] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/misc} time: 00:27:33 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620886/ [5620890] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/blogbench} time: 00:22:23 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620890/ [5620891] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub} time: 00:13:03 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620891/ [5620892] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/dbench} time: 00:52:36 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620892/ [5620894] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/ffsb} time: 00:52:00 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620894/ [5620897] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsstress} time: 00:16:29 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620897/ [5620899] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsx} time: 00:29:53 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620899/ [5620900] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/journal-repair} time: 00:18:08 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620900/ [5620901] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsync-tester} time: 00:11:13 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620901/ [5620903] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush} time: 00:11:40 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620903/ [5620905] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iogen} time: 00:22:10 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620905/ [5620906] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iozone} time: 00:12:53 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620906/ [5620907] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/pjd} time: 00:12:13 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620907/ [5620920] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/misc} time: 00:40:33 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620920/ [5620921] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/kernel_untar_build} time: 00:44:24 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620921/ [5620922] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds_creation_retry} time: 00:07:14 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620922/ [5620923] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/blogbench} time: 00:22:43 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620923/ [5620926] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/openfiletable} time: 00:12:32 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620926/ [5620927] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/ffsb} time: 00:45:44 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620927/ [5620928] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/norstats} time: 00:13:42 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620928/ [5620930] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsstress} time: 00:12:49 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620930/ [5620931] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm} time: 00:11:35 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620931/ [5620932] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:21:12 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620932/ [5620934] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsync-tester} time: 00:12:29 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620934/ [5620936] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/quota} time: 00:11:38 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620936/ [5620937] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/test_o_trunc} time: 00:21:19 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620937/ [5620938] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iogen} time: 00:53:35 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620938/ [5620942] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/pjd} time: 00:13:15 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620942/ [5620944] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:20:43 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620944/ [5620946] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:41:15 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620946/ [5620950] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/direct_io} time: 00:12:52 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620950/ [5620951] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/fs/misc} time: 00:30:25 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620951/ [5620952] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap} time: 00:14:13 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620952/ [5620955] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/blogbench} time: 00:21:33 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620955/ [5620956] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/strays} time: 00:27:05 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620956/ [5620957] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/dbench} time: 00:54:06 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620957/ [5620961] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/ffsb} time: 00:51:38 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620961/ [5620963] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/test_journal_migration} time: 00:17:04 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620963/ [5620964] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsstress} time: 00:16:16 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620964/ [5620966] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsx} time: 00:20:15 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620966/ [5620968] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/fsync-tester} time: 00:11:13 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620968/ [5620971] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iogen} time: 00:54:32 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620971/ [5620972] fs/functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/quota} time: 00:11:46 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620972/ [5620973] fs/workload/{begin clusters/1a2s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} tasks/workunit/suites/iozone} time: 00:13:30 info: http://pulpito.ceph.com/pdonnell-2020-11-14_02:06:06-fs-wip-pdonnell-testing-20201113.235156-distro-basic-smithi/5620973/