2023-10-15T15:07:03.512 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-15T15:07:03.514 INFO:teuthology.results:subset: '111/12000' 2023-10-15T15:07:03.514 INFO:teuthology.results:seed: '4243' 2023-10-15T15:50:00.298 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-15T15:50:00.301 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-10-15T17:08:06.341 INFO:teuthology.results:Tests finished! gathering results... 2023-10-15T17:08:07.319 INFO:scrape:Found 16 jobs 2023-10-15T17:08:07.865 INFO:scrape:Found 2 distinct failure reasons 2023-10-15T17:08:07.866 INFO:scrape:Failure: Command failed (workunit test suites/fsx.sh) on smithi150 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bcf5b23faa7fe51d364c0e949f01773ae6822e01 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2023-10-15T17:08:07.867 INFO:scrape:1 jobs: ['7426838'] 2023-10-15T17:08:07.867 INFO:scrape:suites: ['ignorelist_health', 'objectstore/bluestore-low-osd-mem-target', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-distros/centos_latest', 'tasks/cfuse_workunit_suites_fsx', 'thrashosds-health}'] 2023-10-15T17:08:07.868 INFO:scrape: 2023-10-15T17:08:07.868 INFO:scrape:Failure: Command failed (workunit test kernel_untar_build.sh) on smithi086 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bcf5b23faa7fe51d364c0e949f01773ae6822e01 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' 2023-10-15T17:08:07.868 INFO:scrape:1 jobs: ['7426834'] 2023-10-15T17:08:07.869 INFO:scrape:suites: ['ignorelist_health', 'objectstore/bluestore-comp-snappy', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-distros/centos_latest', 'tasks/cfuse_workunit_kernel_untar_build', 'thrashosds-health}'] 2023-10-15T17:08:07.869 INFO:scrape: 2023-10-15T17:08:07.870 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/ failed: 2 dead: 0 running: 0 waiting: 0 queued: 0 passed: 14 Fail ================================================================= [7426834] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-snappy powercycle/default supported-distros/centos_latest tasks/cfuse_workunit_kernel_untar_build thrashosds-health} ----------------------------------------------------------------- time: 00:33:58 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426834/ log: http://qa-proxy.ceph.com/teuthology/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426834/ sentry: https://sentry.ceph.com/organizations/ceph/?query=52c6c0875692436cb67f9d2a5aaabbe8 Command failed (workunit test kernel_untar_build.sh) on smithi086 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bcf5b23faa7fe51d364c0e949f01773ae6822e01 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' [7426838] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-low-osd-mem-target powercycle/default supported-distros/centos_latest tasks/cfuse_workunit_suites_fsx thrashosds-health} ----------------------------------------------------------------- time: 00:13:57 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426838/ log: http://qa-proxy.ceph.com/teuthology/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426838/ sentry: https://sentry.ceph.com/organizations/ceph/?query=767e36a582544a08bfe9ebeb18bb9ed6 Command failed (workunit test suites/fsx.sh) on smithi150 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bcf5b23faa7fe51d364c0e949f01773ae6822e01 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' Pass ================================================================= [7426831] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-stupid powercycle/default supported-distros/ubuntu_latest tasks/snaps-few-objects thrashosds-health} time: 00:30:14 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426831/ [7426832] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-bitmap powercycle/default supported-distros/centos_latest tasks/snaps-many-objects thrashosds-health} time: 00:26:16 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426832/ [7426833] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-lz4 powercycle/default supported-distros/ubuntu_latest tasks/admin_socket_objecter_requests thrashosds-health} time: 00:19:04 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426833/ [7426835] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zlib powercycle/default supported-distros/ubuntu_latest tasks/cfuse_workunit_misc thrashosds-health} time: 00:40:09 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426835/ [7426836] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zstd powercycle/default supported-distros/centos_latest tasks/cfuse_workunit_suites_ffsb thrashosds-health} time: 00:26:12 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426836/ [7426837] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-hybrid powercycle/default supported-distros/ubuntu_latest tasks/cfuse_workunit_suites_fsstress thrashosds-health} time: 00:18:04 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426837/ [7426839] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-stupid powercycle/default supported-distros/ubuntu_latest tasks/cfuse_workunit_suites_fsync thrashosds-health} time: 00:17:54 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426839/ [7426840] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-bitmap powercycle/default supported-distros/centos_latest tasks/cfuse_workunit_suites_pjd thrashosds-health} time: 00:13:41 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426840/ [7426841] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-lz4 powercycle/default supported-distros/ubuntu_latest tasks/cfuse_workunit_suites_truncate_delay thrashosds-health} time: 00:14:32 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426841/ [7426842] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-snappy powercycle/default supported-distros/centos_latest tasks/rados_api_tests thrashosds-health} time: 00:27:48 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426842/ [7426843] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zlib powercycle/default supported-distros/ubuntu_latest tasks/radosbench thrashosds-health} time: 01:12:38 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426843/ [7426844] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zstd powercycle/default supported-distros/centos_latest tasks/readwrite thrashosds-health} time: 00:15:28 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426844/ [7426845] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-hybrid powercycle/default supported-distros/ubuntu_latest tasks/snaps-few-objects thrashosds-health} time: 00:29:25 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426845/ [7426846] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-low-osd-mem-target powercycle/default supported-distros/centos_latest tasks/snaps-many-objects thrashosds-health} time: 00:26:52 info: http://pulpito.ceph.com/lflores-2023-10-13_22:40:09-powercycle-main-distro-default-smithi/7426846/