2024-01-18T00:51:22.399 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-18T00:51:22.399 INFO:teuthology.results:seed: '7355' 2024-01-18T01:13:53.000 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-18T01:13:53.004 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-01-18T01:47:55.273 INFO:teuthology.results:Tests finished! gathering results... 2024-01-18T01:47:55.638 INFO:scrape:Found 8 jobs 2024-01-18T01:47:56.537 INFO:scrape:Found 2 distinct failure reasons 2024-01-18T01:47:56.538 INFO:scrape:Failure: Command failed on smithi190 with status 130: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0' 2024-01-18T01:47:56.538 INFO:scrape:3 jobs: ['7519869', '7519875', '7519872'] 2024-01-18T01:47:56.538 INFO:scrape:suites intersection: ['cluster/{fixed-3', 'conf/{disable-pool-app}', 'msgr-failures/few', 'openstack}', 'rbd:nbd/{base/install', 'thrashers/default', 'thrashosds-health', 'workloads/rbd_fsx_nbd}'] 2024-01-18T01:47:56.538 INFO:scrape:suites union: ['cluster/{fixed-3', 'conf/{disable-pool-app}', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rbd:nbd/{base/install', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/default', 'thrashosds-health', 'workloads/rbd_fsx_nbd}'] 2024-01-18T01:47:56.538 INFO:scrape: 2024-01-18T01:47:56.538 INFO:scrape:Failure: Command failed (workunit test rbd/rbd-nbd.sh) on smithi190 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c35899e6a2abdc4f8c8e215e5b2fbcdf3dcb8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd-nbd.sh' 2024-01-18T01:47:56.538 INFO:scrape:3 jobs: ['7519876', '7519870', '7519873'] 2024-01-18T01:47:56.538 INFO:scrape:suites intersection: ['cluster/{fixed-3', 'conf/{disable-pool-app}', 'msgr-failures/few', 'openstack}', 'rbd:nbd/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/default', 'thrashosds-health', 'workloads/rbd_nbd}'] 2024-01-18T01:47:56.538 INFO:scrape:suites union: ['cluster/{fixed-3', 'conf/{disable-pool-app}', 'msgr-failures/few', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-stupid', 'openstack}', 'rbd:nbd/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/default', 'thrashosds-health', 'workloads/rbd_nbd}'] 2024-01-18T01:47:56.539 INFO:scrape: 2024-01-18T01:47:56.539 INFO:teuthology.results:Sending results to rraja@redhat.com: Test Run: rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/ failed: 6 dead: 0 running: 0 waiting: 0 queued: 0 passed: 2 Fail ================================================================= [7519869] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} ----------------------------------------------------------------- time: 00:11:38 info: https://pulpito.ceph.com/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519869/ log: http://qa-proxy.ceph.com/teuthology/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519869/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f9c1d0cd65f2406682420f97c9c294d3 Command failed on smithi190 with status 130: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0' [7519870] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd} ----------------------------------------------------------------- time: 00:12:30 info: https://pulpito.ceph.com/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519870/ log: http://qa-proxy.ceph.com/teuthology/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519870/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a7193e757ec540138049e8608cd7a6a4 Command failed (workunit test rbd/rbd-nbd.sh) on smithi161 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c35899e6a2abdc4f8c8e215e5b2fbcdf3dcb8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd-nbd.sh' [7519872] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} ----------------------------------------------------------------- time: 00:11:45 info: https://pulpito.ceph.com/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519872/ log: http://qa-proxy.ceph.com/teuthology/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519872/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5971e6c95eb141cba78637db1f9f92b8 Command failed on smithi175 with status 130: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0' [7519873] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd} ----------------------------------------------------------------- time: 00:12:00 info: https://pulpito.ceph.com/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519873/ log: http://qa-proxy.ceph.com/teuthology/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519873/ sentry: https://sentry.ceph.com/organizations/ceph/?query=43ea3ef43e034e5a80b33b63fcea5431 Command failed (workunit test rbd/rbd-nbd.sh) on smithi186 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c35899e6a2abdc4f8c8e215e5b2fbcdf3dcb8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd-nbd.sh' [7519875] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} ----------------------------------------------------------------- time: 00:08:50 info: https://pulpito.ceph.com/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519875/ log: http://qa-proxy.ceph.com/teuthology/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519875/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ba6cadc1d810489795835ac9e5c2301e Command failed on smithi148 with status 130: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0' [7519876] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd} ----------------------------------------------------------------- time: 00:12:10 info: https://pulpito.ceph.com/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519876/ log: http://qa-proxy.ceph.com/teuthology/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519876/ sentry: https://sentry.ceph.com/organizations/ceph/?query=954162786aa14d409d54a508a4e7fe03 Command failed (workunit test rbd/rbd-nbd.sh) on smithi190 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf9c35899e6a2abdc4f8c8e215e5b2fbcdf3dcb8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd-nbd.sh' Pass ================================================================= [7519871] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous} time: 00:43:25 info: https://pulpito.ceph.com/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519871/ [7519874] rbd:nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous} time: 00:36:56 info: https://pulpito.ceph.com/rraja-2024-01-18_00:51:14-rbd:nbd-wip-ajarr-64063-distro-default-smithi/7519874/