2023-10-21T03:46:38.183 INFO:root:teuthology version: 0.0.1.dev201+g202b180 2023-10-21T03:46:38.185 INFO:teuthology.results:subset: '111/12000' 2023-10-21T03:46:38.185 INFO:teuthology.results:seed: '1721' 2023-10-21T04:30:58.299 INFO:root:teuthology version: 0.0.1.dev201+g202b180 2023-10-21T04:30:58.328 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-10-21T05:56:04.890 INFO:teuthology.results:Tests finished! gathering results... 2023-10-21T05:56:06.157 INFO:scrape:Found 16 jobs 2023-10-21T05:56:06.900 INFO:scrape:Found 2 distinct failure reasons 2023-10-21T05:56:06.901 INFO:scrape:Failure: Command failed (workunit test kernel_untar_build.sh) on smithi050 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f0804064d67a2a63dbabf1801ff452580a050cfc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' 2023-10-21T05:56:06.901 INFO:scrape:1 jobs: ['7433457'] 2023-10-21T05:56:06.902 INFO:scrape:suites: ['ignorelist_health', 'objectstore/bluestore-comp-snappy', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-distros/centos_latest', 'tasks/cfuse_workunit_kernel_untar_build', 'thrashosds-health}'] 2023-10-21T05:56:06.902 INFO:scrape: 2023-10-21T05:56:06.903 INFO:scrape:Failure: Command failed (workunit test suites/fsx.sh) on smithi061 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f0804064d67a2a63dbabf1801ff452580a050cfc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2023-10-21T05:56:06.903 INFO:scrape:1 jobs: ['7433465'] 2023-10-21T05:56:06.904 INFO:scrape:suites: ['ignorelist_health', 'objectstore/bluestore-low-osd-mem-target', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-distros/centos_latest', 'tasks/cfuse_workunit_suites_fsx', 'thrashosds-health}'] 2023-10-21T05:56:06.904 INFO:scrape: 2023-10-21T05:56:06.905 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/ failed: 2 dead: 0 running: 0 waiting: 0 queued: 0 passed: 14 Fail ================================================================= [7433457] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-snappy powercycle/default supported-distros/centos_latest tasks/cfuse_workunit_kernel_untar_build thrashosds-health} ----------------------------------------------------------------- time: 00:38:23 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433457/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433457/ sentry: https://sentry.ceph.com/organizations/ceph/?query=16e64bb5c1294617a29005f1a130d3e6 Command failed (workunit test kernel_untar_build.sh) on smithi050 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f0804064d67a2a63dbabf1801ff452580a050cfc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' [7433465] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-low-osd-mem-target powercycle/default supported-distros/centos_latest tasks/cfuse_workunit_suites_fsx thrashosds-health} ----------------------------------------------------------------- time: 00:14:24 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433465/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433465/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7def48f45c76429bbe52bb6c793b2668 Command failed (workunit test suites/fsx.sh) on smithi061 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f0804064d67a2a63dbabf1801ff452580a050cfc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' Pass ================================================================= [7433452] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-stupid powercycle/default supported-distros/ubuntu_latest tasks/snaps-few-objects thrashosds-health} time: 00:30:28 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433452/ [7433453] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-bitmap powercycle/default supported-distros/centos_latest tasks/snaps-many-objects thrashosds-health} time: 00:26:55 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433453/ [7433455] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-lz4 powercycle/default supported-distros/ubuntu_latest tasks/admin_socket_objecter_requests thrashosds-health} time: 00:18:05 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433455/ [7433459] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zlib powercycle/default supported-distros/ubuntu_latest tasks/cfuse_workunit_misc thrashosds-health} time: 00:48:40 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433459/ [7433461] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zstd powercycle/default supported-distros/centos_latest tasks/cfuse_workunit_suites_ffsb thrashosds-health} time: 00:25:40 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433461/ [7433463] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-hybrid powercycle/default supported-distros/ubuntu_latest tasks/cfuse_workunit_suites_fsstress thrashosds-health} time: 00:16:22 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433463/ [7433467] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-stupid powercycle/default supported-distros/ubuntu_latest tasks/cfuse_workunit_suites_fsync thrashosds-health} time: 00:18:07 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433467/ [7433469] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-bitmap powercycle/default supported-distros/centos_latest tasks/cfuse_workunit_suites_pjd thrashosds-health} time: 00:14:10 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433469/ [7433470] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-lz4 powercycle/default supported-distros/ubuntu_latest tasks/cfuse_workunit_suites_truncate_delay thrashosds-health} time: 00:14:14 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433470/ [7433472] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-snappy powercycle/default supported-distros/centos_latest tasks/rados_api_tests thrashosds-health} time: 00:37:33 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433472/ [7433475] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zlib powercycle/default supported-distros/ubuntu_latest tasks/radosbench thrashosds-health} time: 01:20:55 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433475/ [7433477] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-comp-zstd powercycle/default supported-distros/centos_latest tasks/readwrite thrashosds-health} time: 00:14:54 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433477/ [7433479] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-hybrid powercycle/default supported-distros/ubuntu_latest tasks/snaps-few-objects thrashosds-health} time: 00:30:38 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433479/ [7433481] powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-low-osd-mem-target powercycle/default supported-distros/centos_latest tasks/snaps-many-objects thrashosds-health} time: 00:26:47 info: http://pulpito.ceph.com/yuriw-2023-10-20_18:40:15-powercycle-main-distro-default-smithi/7433481/