2020-11-17T09:26:25.775 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-17T09:26:25.777 INFO:teuthology.results:seed: '4835' 2020-11-17T09:50:55.323 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-17T09:50:55.785 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-11-17T11:46:57.779 INFO:teuthology.results:Tests finished! gathering results... 2020-11-17T11:46:59.058 INFO:scrape:Found 13 jobs 2020-11-17T11:46:59.366 INFO:scrape:Found 1 distinct failure reasons 2020-11-17T11:46:59.367 INFO:scrape:Failure: Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi061 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a0eebba212079365a0362b31ebeb290c1ec5581e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' 2020-11-17T11:46:59.368 INFO:scrape:1 jobs: ['5633445'] 2020-11-17T11:46:59.368 INFO:scrape:suites: ['2-workunit/cfuse_workunit_snaptests}}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/none', 'multifs/yes', 'objectstore-ec/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-17T11:46:59.369 INFO:scrape: 2020-11-17T11:46:59.370 INFO:teuthology.results:Sending results to rraja@redhat.com: Test Run: rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 12 Fail ================================================================= [5633445] fs/thrash/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none multifs/yes objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:11:38 info: http://pulpito.ceph.com/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/5633445/ log: http://qa-proxy.ceph.com/teuthology/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/5633445/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e150ef595caf447aae1f4da1f91dfba5 Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi061 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a0eebba212079365a0362b31ebeb290c1ec5581e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' Pass ================================================================= [5633442] fs/thrash/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none multifs/no objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} time: 02:14:40 info: http://pulpito.ceph.com/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/5633442/ [5633443] fs/thrash/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none multifs/no objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} time: 00:12:46 info: http://pulpito.ceph.com/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/5633443/ [5633444] fs/thrash/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none multifs/no objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:13:20 info: http://pulpito.ceph.com/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/5633444/ [5633446] fs/thrash/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none multifs/yes objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/cfuse_workunit_suites_pjd}} time: 00:13:50 info: http://pulpito.ceph.com/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/5633446/ [5633447] fs/thrash/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none multifs/yes objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:13:54 info: http://pulpito.ceph.com/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/5633447/ [5633448] fs/thrash/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none multifs/yes objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/iozone}} time: 00:14:39 info: http://pulpito.ceph.com/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/5633448/ [5633449] fs/thrash/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none multifs/yes objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:24:29 info: http://pulpito.ceph.com/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/5633449/ [5633450] fs/thrash/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none multifs/yes objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} time: 00:11:50 info: http://pulpito.ceph.com/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/5633450/ [5633451] fs/thrash/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none multifs/yes objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/iozone}} time: 00:14:36 info: http://pulpito.ceph.com/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/5633451/ [5633452] fs/thrash/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none multifs/no objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/cfuse_workunit_trivial_sync}} time: 00:11:08 info: http://pulpito.ceph.com/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/5633452/ [5633453] fs/thrash/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none multifs/no objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/osd 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:24:16 info: http://pulpito.ceph.com/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/5633453/ [5633454] fs/thrash/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none multifs/no objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:17:46 info: http://pulpito.ceph.com/rraja-2020-11-17_08:48:10-fs-wip-ajarr-multifs-thrasher-distro-basic-smithi/5633454/