2020-10-08T10:36:31.925 INFO:root:teuthology version: 1.0.0-2cf4b81 2020-10-08T10:36:31.928 INFO:teuthology.results:subset: '3/4' 2020-10-08T10:36:31.929 INFO:teuthology.results:seed: '6118' 2020-10-08T10:40:39.707 INFO:root:teuthology version: 1.0.0-2cf4b81 2020-10-08T10:40:39.710 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-10-08T22:40:52.226 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2020-10-08T22:40:53.666 INFO:scrape:Found 13 jobs 2020-10-08T22:40:56.831 INFO:scrape:Found 3 distinct failure reasons 2020-10-08T22:40:56.832 INFO:scrape:Failure: Command failed (workunit test fs/test_python.sh) on smithi120 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=39aa5feeb9e47227becb8cc72ddfde3e8f4bea32 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_python.sh' 2020-10-08T22:40:56.833 INFO:scrape:2 jobs: ['5506580', '5506573'] 2020-10-08T22:40:56.833 INFO:scrape:suites intersection: ['clusters/1-mds-4-client-coloc', 'conf/{client', 'fs/basic_functional/{begin', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile', 'osd}', 'overrides/{frag_enable', 'tasks/libcephfs_python}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-10-08T22:40:56.833 INFO:scrape:suites union: ['clusters/1-mds-4-client-coloc', 'conf/{client', 'fs/basic_functional/{begin', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{frag_enable', 'supported-random-distros$/{centos_8}', 'supported-random-distros$/{ubuntu_latest}', 'tasks/libcephfs_python}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-10-08T22:40:56.834 INFO:scrape: 2020-10-08T22:40:56.834 INFO:scrape:Dead: b'2020-10-08T22:38:39.094 DEBUG:teuthology.exit:Finished running handlers' 2020-10-08T22:40:56.834 INFO:scrape:1 jobs: ['5506578'] 2020-10-08T22:40:56.834 INFO:scrape:suites: ['clusters/1-mds-4-client-coloc', 'conf/{client', 'fs/basic_functional/{begin', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{frag_enable', 'supported-random-distros$/{rhel_8}', 'tasks/client-recovery}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-10-08T22:40:56.835 INFO:scrape: 2020-10-08T22:40:56.835 INFO:scrape:Failure: Test failure: test_subvolume_retain_snapshot_invalid_recreate (tasks.cephfs.test_volumes.TestVolumes) 2020-10-08T22:40:56.835 INFO:scrape:1 jobs: ['5506576'] 2020-10-08T22:40:56.836 INFO:scrape:suites: ['clusters/1-mds-4-client-coloc', 'conf/{client', 'fs/basic_functional/{begin', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{frag_enable', 'supported-random-distros$/{rhel_8}', 'tasks/volumes}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-10-08T22:40:56.836 INFO:scrape: 2020-10-08T22:40:56.837 INFO:teuthology.results:Sending results to vshankar@redhat.com: Test Run: vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/ failed: 3 dead: 0 running: 1 waiting: 0 queued: 0 passed: 9 Fail ================================================================= [5506573] fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{ubuntu_latest} tasks/libcephfs_python} ----------------------------------------------------------------- time: 00:08:06 info: http://pulpito.ceph.com/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506573/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506573/ sentry: https://sentry.ceph.com/sentry/sepia/?query=359a2ac9b6224d0b8ba31ebb299dee16 Command failed (workunit test fs/test_python.sh) on smithi167 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=39aa5feeb9e47227becb8cc72ddfde3e8f4bea32 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_python.sh' [5506576] fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{rhel_8} tasks/volumes} ----------------------------------------------------------------- time: 00:45:34 info: http://pulpito.ceph.com/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506576/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506576/ sentry: https://sentry.ceph.com/sentry/sepia/?query=af1884c0ed584d02a137c2e1ed308d1f Test failure: test_subvolume_retain_snapshot_invalid_recreate (tasks.cephfs.test_volumes.TestVolumes) [5506580] fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{centos_8} tasks/libcephfs_python} ----------------------------------------------------------------- time: 00:07:28 info: http://pulpito.ceph.com/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506580/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506580/ sentry: https://sentry.ceph.com/sentry/sepia/?query=4eb76333aa7e4027a5c76804a3e41a80 Command failed (workunit test fs/test_python.sh) on smithi120 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=39aa5feeb9e47227becb8cc72ddfde3e8f4bea32 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_python.sh' Running ================================================================= [5506578] fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{rhel_8} tasks/client-recovery} info: http://pulpito.ceph.com/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506578/ Pass ================================================================= [5506569] fs/basic_workload/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs omap_limit/10000 overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{rhel_8} tasks/libcephfs_interface_tests} time: 00:12:50 info: http://pulpito.ceph.com/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506569/ [5506570] fs/basic_workload/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{centos_8} tasks/cfuse_workunit_suites_fsstress} time: 00:11:46 info: http://pulpito.ceph.com/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506570/ [5506571] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{rhel_8} tasks/cfuse_workunit_suites_pjd} time: 00:11:50 info: http://pulpito.ceph.com/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506571/ [5506572] fs/thrash/{begin ceph-thrash/mon clusters/1-mds-1-client-coloc conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{rhel_8} tasks/cfuse_workunit_snaptests} time: 01:41:35 info: http://pulpito.ceph.com/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506572/ [5506574] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{rhel_8} tasks/cfuse_workunit_suites_pjd} time: 00:11:53 info: http://pulpito.ceph.com/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506574/ [5506575] fs/basic_workload/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{centos_8} tasks/cfuse_workunit_suites_ffsb} time: 01:08:34 info: http://pulpito.ceph.com/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506575/ [5506577] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{ubuntu_latest} tasks/cfuse_workunit_suites_pjd} time: 00:09:58 info: http://pulpito.ceph.com/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506577/ [5506579] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{rhel_8} tasks/cfuse_workunit_suites_pjd} time: 00:12:04 info: http://pulpito.ceph.com/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506579/ [5506581] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{rhel_8} tasks/cfuse_workunit_suites_pjd} time: 00:11:21 info: http://pulpito.ceph.com/vshankar-2020-10-08_10:35:53-fs-wip-cephfs-mirror-vxattr-20201008-020002-distro-basic-smithi/5506581/