Found 71 jobs Not-understood crash indication 2023-12-19T19:20:30.106 INFO:tasks.rgw.c2.client.0.smithi079.stderr:daemon-helper: command crashed with signal 11 Not-understood crash indication 2023-12-19T19:20:35.036 INFO:tasks.rgw.c1.client.1.smithi016.stderr:daemon-helper: command crashed with signal 11 Not-understood crash indication 2023-12-19T20:56:22.079 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi079.stderr: INFO: daemon-helper: command crashed with signal 11 Not-understood crash indication 2023-12-19T20:56:22.085 INFO:tasks.rgw_multisite_tests:tasks.rgw.c1.client.1.smithi016.stderr: INFO: daemon-helper: command crashed with signal 11 Not-understood crash indication Not-understood crash indication 2023-12-19T19:20:30.106 INFO:tasks.rgw.c2.client.0.smithi079.stderr:daemon-helper: command crashed with signal 11 Not-understood crash indication 2023-12-19T19:20:35.036 INFO:tasks.rgw.c1.client.1.smithi016.stderr:daemon-helper: command crashed with signal 11 Not-understood crash indication 2023-12-19T20:56:22.079 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi079.stderr: INFO: daemon-helper: command crashed with signal 11 Not-understood crash indication 2023-12-19T20:56:22.085 INFO:tasks.rgw_multisite_tests:tasks.rgw.c1.client.1.smithi016.stderr: INFO: daemon-helper: command crashed with signal 11 Not-understood crash indication Not-understood crash indication 2023-12-19T19:20:30.106 INFO:tasks.rgw.c2.client.0.smithi079.stderr:daemon-helper: command crashed with signal 11 Not-understood crash indication 2023-12-19T19:20:35.036 INFO:tasks.rgw.c1.client.1.smithi016.stderr:daemon-helper: command crashed with signal 11 Not-understood crash indication 2023-12-19T20:56:22.079 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi079.stderr: INFO: daemon-helper: command crashed with signal 11 Not-understood crash indication 2023-12-19T20:56:22.085 INFO:tasks.rgw_multisite_tests:tasks.rgw.c1.client.1.smithi016.stderr: INFO: daemon-helper: command crashed with signal 11 Not-understood crash indication Not-understood crash indication 2023-12-19T19:28:19.808 INFO:tasks.rgw.client.0.smithi119.stderr:daemon-helper: command crashed with signal 6 Not-understood crash indication Not-understood crash indication 2023-12-19T19:28:19.808 INFO:tasks.rgw.client.0.smithi119.stderr:daemon-helper: command crashed with signal 6 Not-understood crash indication Not-understood crash indication 2023-12-19T19:28:19.808 INFO:tasks.rgw.client.0.smithi119.stderr:daemon-helper: command crashed with signal 6 Not-understood crash indication Not-understood crash indication 2023-12-19T19:28:19.808 INFO:tasks.rgw.client.0.smithi119.stderr:daemon-helper: command crashed with signal 6 Not-understood crash indication Not-understood crash indication 2023-12-19T19:28:19.808 INFO:tasks.rgw.client.0.smithi119.stderr:daemon-helper: command crashed with signal 6 Not-understood crash indication Not-understood crash indication 2023-12-19T19:40:01.920 INFO:tasks.rgw.c1.client.0.smithi001.stderr:daemon-helper: command crashed with signal 11 Not-understood crash indication 2023-12-19T21:03:35.531 INFO:tasks.rgw_multisite_tests:tasks.rgw.c1.client.0.smithi001.stderr: INFO: daemon-helper: command crashed with signal 11 Not-understood crash indication Found 6 distinct failure reasons Failure: rgw multisite test failures 2 jobs: ['7496269', '7496304'] suites intersection: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'rgw/multisite/{clusters', 'supported-random-distro$/{ubuntu_latest}', 'tasks/test_multi}'] suites union: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/three-zones', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'supported-random-distro$/{ubuntu_latest}', 'tasks/test_multi}'] Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fe967764eec] ceph version 19.0.0-202-g41e58cbb (41e58cbbad79bf12af3a2d52e23bc926c8542a59) reef (dev) 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fe966518520] 2: pthread_kill() 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fe967764eec] 6: /usr/lib/ceph/libceph-common.so.2(+0x170046) [0x7fe967765046] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560fbbb50cb1] 8: radosgw-admin(+0x860fba) [0x560fbbb58fba] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x560fbbb5e07b] 10: main() 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fe9664ffd90] 12: __libc_start_main() 13: _start() 1 jobs: ['7496301'] suites: ['cls', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}'] Failure: Command failed (bucket notification tests against different endpoints) on smithi119 with status 1: 'BNTESTS_CONF=/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/bn-tests.client.0.conf /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/bin/python -m nose -s /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py -v -a kafka_test' 1 jobs: ['7496270'] suites: ['bluestore-bitmap', 'centos_latest', 'fixed-2', 'ignore-pg-availability', 'overrides', 'rgw/notifications/{beast', 'tasks/{0-install', 'test_amqp', 'test_kafka', 'test_others}}'] Failure: Command failed (lua tests against rgw) on smithi052 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ceph/src/test/rgw/lua/ && LUATESTS_CONF=./lua-tests.client.0.conf tox -- -v -m 'basic_test or request_test or example_test'" 1 jobs: ['7496267'] suites: ['bluestore-bitmap', 'fixed-2', 'overrides', 'rgw/lua/{beast', 'supported-distros/{centos_latest}', 'tasks/{0-install', 'test_lua}}'] Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7efe409cceec] ceph version 19.0.0-202-g41e58cbb (41e58cbbad79bf12af3a2d52e23bc926c8542a59) reef (dev) 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7efe3f780520] 2: pthread_kill() 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7efe409cceec] 6: /usr/lib/ceph/libceph-common.so.2(+0x170046) [0x7efe409cd046] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5602cc3cacb1] 8: radosgw-admin(+0x860fba) [0x5602cc3d2fba] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x5602cc3d807b] 10: main() 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7efe3f767d90] 12: __libc_start_main() 13: _start() 1 jobs: ['7496280'] suites: ['cls', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}'] Failure: Command failed (workunit test rgw/run-bucket-check.sh) on smithi059 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=41e58cbbad79bf12af3a2d52e23bc926c8542a59 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh' 1 jobs: ['7496318'] suites: ['cls', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}']