2020-06-30T10:23:25.088 INFO:root:teuthology version: 1.0.0-22bcea5 2020-06-30T10:23:25.090 INFO:teuthology.results:subset: '0/4' 2020-06-30T10:23:25.090 INFO:teuthology.results:seed: '8811' 2020-06-30T13:22:03.321 INFO:root:teuthology version: 1.0.0-22bcea5 2020-06-30T13:22:03.335 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-07-01T01:12:24.588 INFO:teuthology.results:Tests finished! gathering results... 2020-07-01T01:12:46.304 INFO:scrape:Found 227 jobs 2020-07-01T01:14:15.236 INFO:scrape:Found 7 distinct failure reasons 2020-07-01T01:14:15.238 INFO:scrape:Failure: Command failed (workunit test suites/pjd.sh) on smithi117 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b12a033c2fbeb415846ddda46d91eeab998e7a4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' 2020-07-01T01:14:15.239 INFO:scrape:3 jobs: ['5191724', '5191589', '5191679'] 2020-07-01T01:14:15.239 INFO:scrape:suites intersection: ['clusters/fixed-2-ucephfs.yaml', 'conf/{client.yaml', 'fs/permission/{begin.yaml', 'mds.yaml', 'mon.yaml', 'mount/fuse.yaml', 'osd.yaml}', 'overrides/{frag_enable.yaml', 'tasks/cfuse_workunit_suites_pjd.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2020-07-01T01:14:15.240 INFO:scrape:suites union: ['clusters/fixed-2-ucephfs.yaml', 'conf/{client.yaml', 'fs/permission/{begin.yaml', 'mds.yaml', 'mon.yaml', 'mount/fuse.yaml', 'objectstore-ec/bluestore-comp.yaml', 'objectstore-ec/bluestore-ec-root.yaml', 'objectstore-ec/filestore-xfs.yaml', 'osd.yaml}', 'overrides/{frag_enable.yaml', 'supported-random-distros$/{centos_latest.yaml}', 'supported-random-distros$/{rhel_latest.yaml}', 'supported-random-distros$/{ubuntu_latest.yaml}', 'tasks/cfuse_workunit_suites_pjd.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2020-07-01T01:14:15.240 INFO:scrape: 2020-07-01T01:14:15.241 INFO:scrape:Failure: "2020-06-30T12:27:36.508941+0000 mon.b (mon.0) 149 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log 2020-07-01T01:14:15.241 INFO:scrape:45 jobs 2020-07-01T01:14:15.242 INFO:scrape:suites intersection: ['conf/{client.yaml', 'mds.yaml', 'mon.yaml', 'mount/fuse.yaml', 'osd.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2020-07-01T01:14:15.243 INFO:scrape:suites union: ['centos_latest.yaml', 'ceph-thrash/mds.yaml', 'ceph-thrash/mon.yaml', 'clusters/1-mds-1-client-coloc.yaml', 'clusters/1-mds-3-client.yaml', 'clusters/1-mds-4-client-coloc.yaml', 'clusters/1a3s-mds-2c-client.yaml', 'clusters/fixed-2-ucephfs.yaml', 'conf/{client.yaml', 'distros/ubuntu_latest.yaml', 'frag_enable.yaml', 'fs/32bits/{begin.yaml', 'fs/basic_functional/{begin.yaml', 'fs/basic_workload/{begin.yaml', 'fs/multiclient/{begin.yaml', 'fs/multifs/{begin.yaml', 'fs/permission/{begin.yaml', 'fs/snaps/{begin.yaml', 'fs/thrash/{begin.yaml', 'fs/traceless/{begin.yaml', 'fs/verify/{begin.yaml', 'inline/no.yaml', 'inline/yes.yaml', 'mds.yaml', 'mon-debug.yaml', 'mon.yaml', 'mount/fuse.yaml', 'msgr-failures/none.yaml', 'msgr-failures/osd-mds-delay.yaml', 'no_client_pidfile.yaml', 'objectstore-ec/bluestore-bitmap.yaml', 'objectstore-ec/bluestore-comp-ec-root.yaml', 'objectstore-ec/bluestore-comp.yaml', 'objectstore-ec/bluestore-ec-root.yaml', 'objectstore-ec/filestore-xfs.yaml', 'objectstore/bluestore-bitmap.yaml', 'objectstore/bluestore-ec-root.yaml', 'omap_limit/10.yaml', 'omap_limit/10000.yaml', 'osd.yaml}', 'overrides/{faked-ino.yaml', 'overrides/{frag_enable.yaml', 'supported-random-distros$/{centos_latest.yaml}', 'supported-random-distros$/{rhel_latest.yaml}', 'supported-random-distros$/{ubuntu_latest.yaml}', 'task/test/{test.yaml}}}', 'tasks/acls-fuse-client.yaml}', 'tasks/asok_dump_tree.yaml}', 'tasks/cfuse_workunit_misc.yaml}', 'tasks/cfuse_workunit_misc_test_o_trunc.yaml}', 'tasks/cfuse_workunit_snaptests.yaml}', 'tasks/cfuse_workunit_suites_blogbench.yaml', 'tasks/cfuse_workunit_suites_blogbench.yaml}', 'tasks/cfuse_workunit_suites_dbench.yaml', 'tasks/cfuse_workunit_suites_dbench.yaml}', 'tasks/cfuse_workunit_suites_ffsb.yaml', 'tasks/cfuse_workunit_suites_fsstress.yaml', 'tasks/cfuse_workunit_suites_fsstress.yaml}', 'tasks/cfuse_workunit_suites_fsx.yaml}', 'tasks/cfuse_workunit_suites_fsync.yaml}', 'tasks/cfuse_workunit_suites_pjd.yaml}', 'tasks/cfuse_workunit_trivial_sync.yaml}', 'tasks/client-readahad.yaml}', 'tasks/damage.yaml}', 'tasks/data-scan.yaml}', 'tasks/failover.yaml}', 'tasks/ior-shared-file.yaml}', 'tasks/libcephfs_interface_tests.yaml}', 'tasks/openfiletable.yaml}', 'tasks/quota.yaml}', 'tasks/sessionmap/{sessionmap.yaml}}', 'tasks/snaptests.yaml}', 'tasks/volume-client/{py/3.yaml', 'traceless/50pc.yaml}', 'validater/valgrind.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2020-07-01T01:14:15.244 INFO:scrape: 2020-07-01T01:14:15.244 INFO:scrape:Failure: Test failure: test_reading_conf_after_resetting_opt (tasks.cephfs.test_cephfs_shell.TestConfReading) 2020-07-01T01:14:15.244 INFO:scrape:2 jobs: ['5191663', '5191548'] 2020-07-01T01:14:15.245 INFO:scrape:suites intersection: ['clusters/1-mds-4-client-coloc.yaml', 'conf/{client.yaml', 'fs/basic_functional/{begin.yaml', 'mds.yaml', 'mon.yaml', 'mount/fuse.yaml', 'no_client_pidfile.yaml', 'osd.yaml}', 'overrides/{frag_enable.yaml', 'supported-random-distros$/{centos_latest.yaml}', 'tasks/cephfs-shell.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2020-07-01T01:14:15.245 INFO:scrape:suites union: ['clusters/1-mds-4-client-coloc.yaml', 'conf/{client.yaml', 'fs/basic_functional/{begin.yaml', 'mds.yaml', 'mon.yaml', 'mount/fuse.yaml', 'no_client_pidfile.yaml', 'objectstore/bluestore-bitmap.yaml', 'objectstore/bluestore-ec-root.yaml', 'osd.yaml}', 'overrides/{frag_enable.yaml', 'supported-random-distros$/{centos_latest.yaml}', 'tasks/cephfs-shell.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2020-07-01T01:14:15.246 INFO:scrape: 2020-07-01T01:14:15.246 INFO:scrape:Dead: b'2020-06-30T22:31:30.284 DEBUG:teuthology.exit:Finished running handlers' 2020-07-01T01:14:15.247 INFO:scrape:7 jobs: ['5191525', '5191685', '5191716', '5191584', '5191553', '5191620', '5191649'] 2020-07-01T01:14:15.247 INFO:scrape:suites intersection: ['1-client.yaml', '2-upgrade.yaml', 'clusters/1-mds-2-client-micro.yaml', 'conf/{client.yaml', 'mds.yaml', 'mon.yaml', 'osd.yaml}', 'overrides/{frag_enable.yaml', 'pg-warn.yaml', 'tasks/{0-mimic.yaml', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2020-07-01T01:14:15.248 INFO:scrape:suites union: ['1-client.yaml', '2-upgrade.yaml', '3-client-upgrade.yaml', '3-compat_client/mimic.yaml}}', '3-compat_client/no.yaml}}', '3-verify.yaml}', '4-compat_client.yaml', '5-client-sanity.yaml}}', 'clusters/1-mds-2-client-micro.yaml', 'conf/{client.yaml', 'fs/upgrade/featureful_client/old_client/{bluestore-bitmap.yaml', 'fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap.yaml', 'fs/upgrade/volumes/import-legacy/{bluestore-bitmap.yaml', 'mds.yaml', 'mon.yaml', 'multimds/no.yaml', 'multimds/yes.yaml', 'osd.yaml}', 'overrides/{frag_enable.yaml', 'pg-warn.yaml', 'tasks/{0-mimic.yaml', 'ubuntu_18.04.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2020-07-01T01:14:15.248 INFO:scrape: 2020-07-01T01:14:15.249 INFO:scrape:Failure: Command failed (workunit test libcephfs/test.sh) on smithi022 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b12a033c2fbeb415846ddda46d91eeab998e7a4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/libcephfs/test.sh' 2020-07-01T01:14:15.249 INFO:scrape:2 jobs: ['5191561', '5191652'] 2020-07-01T01:14:15.249 INFO:scrape:suites intersection: ['clusters/fixed-2-ucephfs.yaml', 'conf/{client.yaml', 'fs/basic_workload/{begin.yaml', 'inline/yes.yaml', 'mds.yaml', 'mon.yaml', 'mount/fuse.yaml', 'omap_limit/10000.yaml', 'osd.yaml}', 'overrides/{frag_enable.yaml', 'supported-random-distros$/{centos_latest.yaml}', 'tasks/libcephfs_interface_tests.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2020-07-01T01:14:15.250 INFO:scrape:suites union: ['clusters/fixed-2-ucephfs.yaml', 'conf/{client.yaml', 'fs/basic_workload/{begin.yaml', 'inline/yes.yaml', 'mds.yaml', 'mon.yaml', 'mount/fuse.yaml', 'objectstore-ec/bluestore-bitmap.yaml', 'objectstore-ec/bluestore-comp.yaml', 'omap_limit/10000.yaml', 'osd.yaml}', 'overrides/{frag_enable.yaml', 'supported-random-distros$/{centos_latest.yaml}', 'tasks/libcephfs_interface_tests.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2020-07-01T01:14:15.250 INFO:scrape: 2020-07-01T01:14:15.251 INFO:scrape:Failure: Test failure: test_ops_throttle (tasks.cephfs.test_strays.TestStrays) 2020-07-01T01:14:15.251 INFO:scrape:1 jobs: ['5191730'] 2020-07-01T01:14:15.252 INFO:scrape:suites: ['clusters/1-mds-4-client-coloc.yaml', 'conf/{client.yaml', 'fs/basic_functional/{begin.yaml', 'mds.yaml', 'mon.yaml', 'mount/fuse.yaml', 'no_client_pidfile.yaml', 'objectstore/bluestore-ec-root.yaml', 'osd.yaml}', 'overrides/{frag_enable.yaml', 'supported-random-distros$/{ubuntu_latest.yaml}', 'tasks/strays.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2020-07-01T01:14:15.252 INFO:scrape: 2020-07-01T01:14:15.252 INFO:scrape:Failure: Test failure: test_join_fs_drop (tasks.cephfs.test_failover.TestClusterAffinity) 2020-07-01T01:14:15.253 INFO:scrape:1 jobs: ['5191610'] 2020-07-01T01:14:15.253 INFO:scrape:suites: ['clusters/1a3s-mds-2c-client.yaml', 'conf/{client.yaml', 'fs/multifs/{begin.yaml', 'mds.yaml', 'mon-debug.yaml', 'mon.yaml', 'mount/fuse.yaml', 'objectstore-ec/bluestore-comp.yaml', 'osd.yaml}', 'overrides/{frag_enable.yaml', 'supported-random-distros$/{centos_latest.yaml}', 'tasks/failover.yaml}', 'whitelist_health.yaml', 'whitelist_wrongly_marked_down.yaml}'] 2020-07-01T01:14:15.254 INFO:scrape: