2021-12-10T13:23:29.321 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-10T13:23:29.323 INFO:teuthology.results:subset: '24491/100000' 2021-12-10T13:23:29.324 INFO:teuthology.results:seed: '4543' 2021-12-10T13:42:30.399 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-10T13:42:30.435 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-12-10T15:20:37.796 INFO:teuthology.results:Tests finished! gathering results... 2021-12-10T15:20:39.027 INFO:scrape:Found 15 jobs 2021-12-10T15:20:39.030 INFO:scrape:Found 2 distinct failure reasons 2021-12-10T15:20:39.031 INFO:scrape:Failure: Command failed (workunit test suites/ffsb.sh) on smithi001 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e714d6effbcf6bc686fac2d23cc159b5a013c11b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2021-12-10T15:20:39.031 INFO:scrape:1 jobs: ['6552543'] 2021-12-10T15:20:39.031 INFO:scrape:suites: ['objectstore/bluestore-low-osd-mem-target', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/centos_8.stream', 'tasks/cfuse_workunit_suites_ffsb', 'thrashosds-health', 'whitelist_health}'] 2021-12-10T15:20:39.032 INFO:scrape: 2021-12-10T15:20:39.032 INFO:scrape:Timeout 120 running ceph --cluster ceph pg dump --format=json 2021-12-10T15:20:39.033 INFO:scrape:1 jobs: ['6552541'] 2021-12-10T15:20:39.033 INFO:scrape:suites: ['objectstore/bluestore-comp-zstd', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/rhel_8', 'tasks/cfuse_workunit_kernel_untar_build', 'thrashosds-health', 'whitelist_health}'] 2021-12-10T15:20:39.034 INFO:scrape: 2021-12-10T15:20:39.034 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/ failed: 2 dead: 0 running: 0 waiting: 0 queued: 0 passed: 13 Fail ================================================================= [6552541] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zstd powercycle/default supported-all-distro/rhel_8 tasks/cfuse_workunit_kernel_untar_build thrashosds-health whitelist_health} ----------------------------------------------------------------- time: 01:37:19 info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552541/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552541/ Command failed on smithi060 with status 124: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' [6552543] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-low-osd-mem-target powercycle/default supported-all-distro/centos_8.stream tasks/cfuse_workunit_suites_ffsb thrashosds-health whitelist_health} ----------------------------------------------------------------- time: 00:56:52 info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552543/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552543/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e7f58e5e215943e3a70fd75762861522 Command failed (workunit test suites/ffsb.sh) on smithi001 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e714d6effbcf6bc686fac2d23cc159b5a013c11b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' Pass ================================================================= [6552531] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zlib powercycle/default supported-all-distro/centos_8.stream tasks/cfuse_workunit_suites_fsx thrashosds-health whitelist_health} time: 00:28:41 info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552531/ [6552532] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zstd powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_suites_fsync thrashosds-health whitelist_health} time: 00:13:55 info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552532/ [6552533] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-hybrid powercycle/default supported-all-distro/rhel_8 tasks/cfuse_workunit_suites_pjd thrashosds-health whitelist_health} time: 00:19:42 info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552533/ [6552534] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-low-osd-mem-target powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_suites_truncate_delay thrashosds-health whitelist_health} time: 00:09:40 info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552534/ [6552535] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-stupid powercycle/default supported-all-distro/centos_8.stream tasks/rados_api_tests thrashosds-health whitelist_health} time: 00:30:33 info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552535/ [6552536] powercycle/osd/{clusters/3osd-1per-target objectstore/filestore-xfs powercycle/default supported-all-distro/centos_8 tasks/radosbench thrashosds-health whitelist_health} time: 01:08:30 info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552536/ [6552537] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-bitmap powercycle/default supported-all-distro/rhel_8 tasks/readwrite thrashosds-health whitelist_health} time: 00:19:02 info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552537/ [6552538] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-lz4 powercycle/default supported-all-distro/ubuntu_latest tasks/snaps-few-objects thrashosds-health whitelist_health} time: 00:29:05 info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552538/ [6552539] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-snappy powercycle/default supported-all-distro/centos_8.stream tasks/snaps-many-objects thrashosds-health whitelist_health} time: 00:23:29 info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552539/ [6552540] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zlib powercycle/default supported-all-distro/centos_8 tasks/admin_socket_objecter_requests thrashosds-health whitelist_health} time: 00:15:41 info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552540/ [6552542] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-hybrid powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_misc thrashosds-health whitelist_health} time: 00:44:46 info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552542/ [6552544] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-stupid powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_suites_fsstress thrashosds-health whitelist_health} time: 00:19:12 info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552544/ [6552545] powercycle/osd/{clusters/3osd-1per-target objectstore/filestore-xfs powercycle/default supported-all-distro/rhel_8 tasks/cfuse_workunit_suites_fsx thrashosds-health whitelist_health} time: 00:42:43 info: http://pulpito.ceph.com/teuthology-2021-12-08_11:15:03-powercycle-master-distro-default-smithi/6552545/