2023-12-08T22:28:02.760 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-08T22:28:02.761 INFO:teuthology.results:subset: '17/99' 2023-12-08T22:28:02.761 INFO:teuthology.results:seed: '2777' 2023-12-08T22:40:17.614 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-08T22:40:17.618 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-12-09T10:41:14.083 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2023-12-09T10:41:15.918 INFO:scrape:Found 23 jobs 2023-12-09T10:41:16.068 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-87-gcb2aea62 (cb2aea6220668cae9352bb90548c3efd60ccb9d3) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7ff33f9805db] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x180799) [0x7ff33f980799] 2023-12-09T10:41:16.068 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2023-12-09T10:41:16.119 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-87-gcb2aea62 (cb2aea6220668cae9352bb90548c3efd60ccb9d3) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7ff33f9805db] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x180799) [0x7ff33f980799] 2023-12-09T10:41:16.120 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2023-12-09T10:41:16.136 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-87-gcb2aea62 (cb2aea6220668cae9352bb90548c3efd60ccb9d3) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7ff33f9805db] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x180799) [0x7ff33f980799] 2023-12-09T10:41:16.136 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2023-12-09T10:41:16.151 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-87-gcb2aea62 (cb2aea6220668cae9352bb90548c3efd60ccb9d3) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7ff33f9805db] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x180799) [0x7ff33f980799] 2023-12-09T10:41:16.151 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2023-12-09T10:41:16.165 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-87-gcb2aea62 (cb2aea6220668cae9352bb90548c3efd60ccb9d3) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7ff33f9805db] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x180799) [0x7ff33f980799] 2023-12-09T10:41:16.166 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2023-12-09T10:41:17.019 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-87-gcb2aea62 (cb2aea6220668cae9352bb90548c3efd60ccb9d3) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fa5896bd594] , 2: /usr/lib/ceph/libceph-common.so.2(+0x181748) [0x7fa5896bd748] 2023-12-09T10:41:17.020 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2023-12-09T10:41:17.039 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-87-gcb2aea62 (cb2aea6220668cae9352bb90548c3efd60ccb9d3) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fa5896bd594] , 2: /usr/lib/ceph/libceph-common.so.2(+0x181748) [0x7fa5896bd748] 2023-12-09T10:41:17.039 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2023-12-09T10:41:17.170 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-87-gcb2aea62 (cb2aea6220668cae9352bb90548c3efd60ccb9d3) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7ff273b805db] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x180799) [0x7ff273b80799] 2023-12-09T10:41:17.171 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2023-12-09T10:41:17.202 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-87-gcb2aea62 (cb2aea6220668cae9352bb90548c3efd60ccb9d3) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7ff273b805db] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x180799) [0x7ff273b80799] 2023-12-09T10:41:17.202 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2023-12-09T10:41:17.591 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-87-gcb2aea62 (cb2aea6220668cae9352bb90548c3efd60ccb9d3) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f242db805db] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x180799) [0x7f242db80799] 2023-12-09T10:41:17.592 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2023-12-09T10:41:17.625 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-87-gcb2aea62 (cb2aea6220668cae9352bb90548c3efd60ccb9d3) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f242db805db] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x180799) [0x7f242db80799] 2023-12-09T10:41:17.625 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2023-12-09T10:41:17.775 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-87-gcb2aea62 (cb2aea6220668cae9352bb90548c3efd60ccb9d3) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fb03e7805db] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x180799) [0x7fb03e780799] 2023-12-09T10:41:17.775 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2023-12-09T10:41:17.791 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-87-gcb2aea62 (cb2aea6220668cae9352bb90548c3efd60ccb9d3) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fb03e7805db] , 2: /usr/lib64/ceph/libceph-common.so.2(+0x180799) [0x7fb03e780799] 2023-12-09T10:41:17.792 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2023-12-09T10:41:17.799 INFO:scrape:Found 3 distinct failure reasons 2023-12-09T10:41:17.800 INFO:scrape:Assertion: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7ff33f9805db] 2023-12-09T10:41:17.800 INFO:scrape:1 jobs: ['7485139'] 2023-12-09T10:41:17.800 INFO:scrape:suites: ['clusters/{fixed-3', 'conf/{disable-pool-app}', 'data-pool/none', 'extra-conf/permit-partial-discard', 'min-compat-client/default', 'msgr-failures/few', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rbd/librbd/{cache/writeback', 'supported-random-distro$/{centos_latest}', 'workloads/c_api_tests_with_journaling}'] 2023-12-09T10:41:17.800 INFO:scrape: 2023-12-09T10:41:17.800 INFO:scrape:Failure: Command failed (workunit test rbd/test_librbd.sh) on smithi003 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cb2aea6220668cae9352bb90548c3efd60ccb9d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 VALGRIND=\'--tool=memcheck --leak-check=full\' adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' 2023-12-09T10:41:17.800 INFO:scrape:7 jobs: ['7485136', '7485138', '7485142', '7485135', '7485133', '7485131', '7485119'] 2023-12-09T10:41:17.800 INFO:scrape:suites intersection: ['conf/{disable-pool-app}', 'openstack}'] 2023-12-09T10:41:17.801 INFO:scrape:suites union: ['centos_latest', 'clusters/{fixed-1', 'clusters/{fixed-3', 'conf/{disable-pool-app}', 'data-pool/ec', 'data-pool/none', 'data-pool/replicated', 'extra-conf/copy-on-read', 'extra-conf/none', 'extra-conf/permit-partial-discard', 'min-compat-client/default', 'msgr-failures/few', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'openstack}', 'rbd/librbd/{cache/writeback', 'rbd/valgrind/{base/install', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{ubuntu_latest}', 'validator/memcheck', 'workloads/c_api_tests_with_defaults}', 'workloads/c_api_tests_with_journaling}', 'workloads/c_api_tests}'] 2023-12-09T10:41:17.801 INFO:scrape: 2023-12-09T10:41:17.801 INFO:scrape:Dead: b'2023-12-09T10:41:15.351 DEBUG:teuthology.exit:Finished running handlers' 2023-12-09T10:41:17.801 INFO:scrape:ceph version 19.0.0-87-gcb2aea62 (cb2aea6220668cae9352bb90548c3efd60ccb9d3) reef (dev) 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f306ef0f520] 2: pthread_kill() 3: raise() 4: abort() 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x17b) [0x560ef4f28991] 6: (KernelDevice::_aio_thread()+0x9bc) [0x560ef581141c] 7: ceph-osd(+0x15d3cd1) [0x560ef5811cd1] 8: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f306ef61b43] 9: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f306eff3a00] 2023-12-09T10:41:17.801 INFO:scrape:1 jobs: ['7485127'] 2023-12-09T10:41:17.801 INFO:scrape:suites: ['clients/mirror', 'cluster/{2-node', 'conf/{disable-pool-app}', 'msgr-failures/few', 'objectstore/bluestore-stupid', 'openstack}', 'policy/none', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-snapshot-stress-workunit-minimum}'] 2023-12-09T10:41:17.801 INFO:scrape: 2023-12-09T10:41:17.802 INFO:teuthology.results:Sending results to idryomov@gmail.com: Test Run: dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/ failed: 8 dead: 0 running: 1 waiting: 0 queued: 0 passed: 14 Fail ================================================================= [7485119] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} conf/{disable-pool-app} data-pool/none extra-conf/permit-partial-discard min-compat-client/default msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_latest} workloads/c_api_tests_with_journaling} ----------------------------------------------------------------- time: 00:11:10 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485119/ log: http://qa-proxy.ceph.com/teuthology/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485119/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d9e4aef7acc04bb296acba32f8d073fb Command failed (workunit test rbd/test_librbd.sh) on smithi160 with status 134: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cb2aea6220668cae9352bb90548c3efd60ccb9d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=125 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' [7485131] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} conf/{disable-pool-app} data-pool/ec extra-conf/none min-compat-client/default msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} workloads/c_api_tests_with_journaling} ----------------------------------------------------------------- time: 00:11:06 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485131/ log: http://qa-proxy.ceph.com/teuthology/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485131/ sentry: https://sentry.ceph.com/organizations/ceph/?query=271d5a6a859045be9bb9b8a82540b224 Command failed (workunit test rbd/test_librbd.sh) on smithi194 with status 134: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cb2aea6220668cae9352bb90548c3efd60ccb9d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=125 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' [7485133] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} conf/{disable-pool-app} objectstore/bluestore-comp-zlib validator/memcheck workloads/c_api_tests} ----------------------------------------------------------------- time: 01:23:09 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485133/ log: http://qa-proxy.ceph.com/teuthology/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485133/ sentry: https://sentry.ceph.com/organizations/ceph/?query=99207188e0f9485dbffc014ed1bf914b Command failed (workunit test rbd/test_librbd.sh) on smithi136 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cb2aea6220668cae9352bb90548c3efd60ccb9d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=1 VALGRIND=\'-- tool=memcheck --leak-check=full\' adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' [7485135] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} conf/{disable-pool-app} data-pool/replicated extra-conf/copy-on-read min-compat-client/default msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_latest} workloads/c_api_tests_with_journaling} ----------------------------------------------------------------- time: 00:11:42 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485135/ log: http://qa-proxy.ceph.com/teuthology/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485135/ sentry: https://sentry.ceph.com/organizations/ceph/?query=be76426e10b840db98cd9abc131ea4ff Command failed (workunit test rbd/test_librbd.sh) on smithi204 with status 134: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cb2aea6220668cae9352bb90548c3efd60ccb9d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=125 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' [7485136] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} conf/{disable-pool-app} objectstore/bluestore-comp-zstd validator/memcheck workloads/c_api_tests_with_defaults} ----------------------------------------------------------------- time: 01:26:41 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485136/ log: http://qa-proxy.ceph.com/teuthology/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485136/ sentry: https://sentry.ceph.com/organizations/ceph/?query=11cdf597cfeb4f8ba30d477b9b1d4abb Command failed (workunit test rbd/test_librbd.sh) on smithi003 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cb2aea6220668cae9352bb90548c3efd60ccb9d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 VALGRIND=\'-- tool=memcheck --leak-check=full\' adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' [7485138] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} conf/{disable-pool-app} objectstore/bluestore-hybrid validator/memcheck workloads/c_api_tests_with_journaling} ----------------------------------------------------------------- time: 00:55:56 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485138/ log: http://qa-proxy.ceph.com/teuthology/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485138/ sentry: https://sentry.ceph.com/organizations/ceph/?query=37fed156a4344d17bba0f1d32e6148a2 Command failed (workunit test rbd/test_librbd.sh) on smithi064 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cb2aea6220668cae9352bb90548c3efd60ccb9d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=125 VALGRIND=\'-- tool=memcheck --leak-check=full\' adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' [7485139] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} conf/{disable-pool-app} data-pool/none extra-conf/permit-partial-discard min-compat-client/default msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} workloads/c_api_tests_with_journaling} ----------------------------------------------------------------- time: 00:13:04 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485139/ log: http://qa-proxy.ceph.com/teuthology/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485139/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4c81522176ac4091b2fc99cbb8595d17 Command failed (workunit test rbd/test_librbd.sh) on smithi175 with status 134: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cb2aea6220668cae9352bb90548c3efd60ccb9d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=125 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' [7485142] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} conf/{disable-pool-app} data-pool/ec extra-conf/none min-compat-client/default msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} workloads/c_api_tests_with_journaling} ----------------------------------------------------------------- time: 00:13:14 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485142/ log: http://qa-proxy.ceph.com/teuthology/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485142/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4852f007cbf04677a9f7f00965f433d3 Command failed (workunit test rbd/test_librbd.sh) on smithi131 with status 134: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cb2aea6220668cae9352bb90548c3efd60ccb9d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=125 adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' Running ================================================================= [7485127] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum} info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485127/ Pass ================================================================= [7485107] rbd/maintenance/{base/install clusters/{fixed-3 openstack} conf/{disable-pool-app} objectstore/bluestore-low-osd-mem-target qemu/xfstests supported-random-distro$/{ubuntu_latest} workloads/rebuild_object_map} time: 01:48:49 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485107/ [7485109] rbd/nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous} time: 00:40:19 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485109/ [7485111] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} features/format-1 msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} workloads/rbd_cli_generic} time: 00:31:34 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485111/ [7485113] rbd/migration/{1-base/install 2-clusters/{fixed-3 openstack} 3-objectstore/bluestore-comp-lz4 4-supported-random-distro$/{centos_latest} 5-data-pool/replicated 6-prepare/raw-file 7-io-workloads/qemu_xfstests 8-migrate-workloads/execute 9-cleanup/cleanup conf/{disable-pool-app}} time: 00:57:14 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485113/ [7485115] rbd/cli/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} data-pool/ec features/defaults msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} workloads/rbd_cli_generic} time: 00:31:21 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485115/ [7485117] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-config-key} time: 00:36:53 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485117/ [7485121] rbd/basic/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_latest} tasks/rbd_api_tests_old_format} time: 00:21:47 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485121/ [7485123] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} features/format-1 msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} workloads/rbd_cli_import_export} time: 00:10:52 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485123/ [7485125] rbd/pwl-cache/tmpfs/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{ubuntu_latest} 4-cache-path 5-cache-mode/rwl 6-cache-size/1G 7-workloads/qemu_xfstests conf/{disable-pool-app}} time: 01:23:02 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485125/ [7485129] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} conf/{disable-pool-app} objectstore/bluestore-comp-zlib validator/memcheck workloads/rbd_mirror} time: 00:52:44 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485129/ [7485134] rbd/pwl-cache/tmpfs/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{centos_latest} 4-cache-path 5-cache-mode/ssd 6-cache-size/5G 7-workloads/qemu_xfstests conf/{disable-pool-app}} time: 01:03:11 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485134/ [7485137] rbd/singleton-bluestore/{all/issue-20295 conf/{disable-pool-app} objectstore/bluestore-comp-snappy openstack supported-random-distro$/{centos_latest}} time: 00:15:32 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485137/ [7485140] rbd/librbd/{cache/none clusters/{fixed-3 openstack} conf/{disable-pool-app} data-pool/none extra-conf/permit-partial-discard min-compat-client/default msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} workloads/c_api_tests} time: 00:51:39 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485140/ [7485141] rbd/nbd/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous} time: 00:32:23 info: https://pulpito.ceph.com/dis-2023-12-08_21:27:07-rbd-wip-dis-testing-distro-default-smithi/7485141/