2024-01-18T02:00:05.917 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-18T02:00:05.917 INFO:teuthology.results:seed: '3124' 2024-01-18T02:10:52.157 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-18T02:10:52.160 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-01-18T02:58:55.737 INFO:teuthology.results:Tests finished! gathering results... 2024-01-18T02:58:55.990 INFO:scrape:Found 8 jobs 2024-01-18T02:58:56.862 INFO:scrape:Found 2 distinct failure reasons 2024-01-18T02:58:56.863 INFO:scrape:Failure: Command failed on smithi194 with status 130: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0' 2024-01-18T02:58:56.863 INFO:scrape:3 jobs: ['7519905', '7519902', '7519908'] 2024-01-18T02:58:56.863 INFO:scrape:suites intersection: ['cluster/{fixed-3', 'conf/{disable-pool-app}', 'msgr-failures/few', 'openstack}', 'rbd:nbd/{base/install', 'thrashers/default', 'thrashosds-health', 'workloads/rbd_fsx_nbd}'] 2024-01-18T02:58:56.863 INFO:scrape:suites union: ['cluster/{fixed-3', 'conf/{disable-pool-app}', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rbd:nbd/{base/install', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/default', 'thrashosds-health', 'workloads/rbd_fsx_nbd}'] 2024-01-18T02:58:56.863 INFO:scrape: 2024-01-18T02:58:56.863 INFO:scrape:Failure: Command failed (workunit test rbd/rbd-nbd.sh) on smithi156 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c35899e6a2abdc4f8c8e215e5b2fbcdf3dcb8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd-nbd.sh' 2024-01-18T02:58:56.863 INFO:scrape:3 jobs: ['7519909', '7519906', '7519903'] 2024-01-18T02:58:56.864 INFO:scrape:suites intersection: ['cluster/{fixed-3', 'conf/{disable-pool-app}', 'msgr-failures/few', 'openstack}', 'rbd:nbd/{base/install', 'thrashers/default', 'thrashosds-health', 'workloads/rbd_nbd}'] 2024-01-18T02:58:56.864 INFO:scrape:suites union: ['cluster/{fixed-3', 'conf/{disable-pool-app}', 'msgr-failures/few', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-stupid', 'openstack}', 'rbd:nbd/{base/install', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/default', 'thrashosds-health', 'workloads/rbd_nbd}'] 2024-01-18T02:58:56.864 INFO:scrape: 2024-01-18T02:58:56.864 INFO:teuthology.results:Sending results to rraja@redhat.com: Test Run: rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/ failed: 6 dead: 0 running: 0 waiting: 0 queued: 0 passed: 2 Fail ================================================================= [7519902] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} ----------------------------------------------------------------- time: 00:08:51 info: https://pulpito.ceph.com/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519902/ log: http://qa-proxy.ceph.com/teuthology/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519902/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f98b31302f9c4d9683d64c3c67c02c86 Command failed on smithi204 with status 130: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0' [7519903] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_nbd} ----------------------------------------------------------------- time: 00:09:35 info: https://pulpito.ceph.com/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519903/ log: http://qa-proxy.ceph.com/teuthology/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519903/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c7780233745a4b0d938d836c50431985 Command failed (workunit test rbd/rbd-nbd.sh) on smithi170 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c35899e6a2abdc4f8c8e215e5b2fbcdf3dcb8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd-nbd.sh' [7519905] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} ----------------------------------------------------------------- time: 00:11:18 info: https://pulpito.ceph.com/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519905/ log: http://qa-proxy.ceph.com/teuthology/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519905/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c53f408419d646b6a1b2b594ca5436f4 Command failed on smithi194 with status 130: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0' [7519906] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd} ----------------------------------------------------------------- time: 00:12:02 info: https://pulpito.ceph.com/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519906/ log: http://qa-proxy.ceph.com/teuthology/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519906/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ec4e16a099044c60bad4deda31c71cf9 Command failed (workunit test rbd/rbd-nbd.sh) on smithi190 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c35899e6a2abdc4f8c8e215e5b2fbcdf3dcb8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd-nbd.sh' [7519908] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} ----------------------------------------------------------------- time: 00:08:40 info: https://pulpito.ceph.com/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519908/ log: http://qa-proxy.ceph.com/teuthology/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519908/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c94b6a9eb2154e01bf0191fe261df6d8 Command failed on smithi123 with status 130: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0' [7519909] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_nbd} ----------------------------------------------------------------- time: 00:09:46 info: https://pulpito.ceph.com/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519909/ log: http://qa-proxy.ceph.com/teuthology/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519909/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9628794226514f338e92a4786cadd119 Command failed (workunit test rbd/rbd-nbd.sh) on smithi156 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c35899e6a2abdc4f8c8e215e5b2fbcdf3dcb8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd-nbd.sh' Pass ================================================================= [7519904] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous} time: 00:44:12 info: https://pulpito.ceph.com/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519904/ [7519907] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous} time: 00:30:59 info: https://pulpito.ceph.com/rraja-2024-01-18_01:59:30-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519907/