2024-03-18T11:43:45.074 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-03-18T11:43:45.074 INFO:teuthology.results:subset: '0/1' 2024-03-18T11:43:45.074 INFO:teuthology.results:seed: '1663' 2024-03-18T12:25:17.643 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-03-18T12:25:17.648 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-03-18T15:09:29.556 INFO:teuthology.results:Tests finished! gathering results... 2024-03-18T15:09:34.762 INFO:scrape:Found 71 jobs 2024-03-18T15:09:36.099 WARNING:scrape:Not-understood crash indication 2024-03-18T13:25:19.954 INFO:tasks.rgw.c2.client.0.smithi143.stderr:daemon-helper: command crashed with signal 11 2024-03-18T15:09:36.099 WARNING:scrape:Not-understood crash indication 2024-03-18T14:30:37.300 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi143.stderr: INFO: daemon-helper: command crashed with signal 11 2024-03-18T15:09:36.099 WARNING:scrape:Not-understood crash indication 2024-03-18T15:09:45.054 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604921/teuthology.log 2024-03-18T15:09:45.055 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604921/teuthology.log 2024-03-18T15:09:45.055 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604921/teuthology.log 2024-03-18T15:09:45.055 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604921/teuthology.log 2024-03-18T15:09:45.055 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604921/teuthology.log 2024-03-18T15:09:45.055 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604921/teuthology.log 2024-03-18T15:09:45.055 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604921/teuthology.log 2024-03-18T15:09:45.056 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604921/teuthology.log 2024-03-18T15:09:45.056 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604921/teuthology.log 2024-03-18T15:09:45.056 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604921/teuthology.log 2024-03-18T15:09:45.056 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604921/teuthology.log 2024-03-18T15:09:45.056 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604921/teuthology.log 2024-03-18T15:09:45.056 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604921/teuthology.log 2024-03-18T15:10:03.593 INFO:scrape:Found 27 distinct failure reasons 2024-03-18T15:10:03.594 INFO:scrape:Failure: rgw multisite test failures 2024-03-18T15:10:03.594 INFO:scrape:2 jobs: ['7604914', '7604879'] 2024-03-18T15:10:03.594 INFO:scrape:suites intersection: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'rgw/multisite/{clusters', 'tasks/test_multi}'] 2024-03-18T15:10:03.594 INFO:scrape:suites union: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/three-zones', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/test_multi}'] 2024-03-18T15:10:03.594 INFO:scrape: 2024-03-18T15:10:03.595 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f1b34b80d4a] 2024-03-18T15:10:03.595 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib64/libc.so.6(+0x54db0) [0x7f1b33a54db0] 2: /lib64/libc.so.6(+0xa154c) [0x7f1b33aa154c] 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f1b34b80d4a] 6: /usr/lib64/ceph/libceph-common.so.2(+0x180eae) [0x7f1b34b80eae] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a22d1e1441] 8: radosgw-admin(+0x7cd44a) [0x55a22d1e944a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x55a22d1edc5b] 10: main() 11: /lib64/libc.so.6(+0x3feb0) [0x7f1b33a3feb0] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.595 INFO:scrape:1 jobs: ['7604907'] 2024-03-18T15:10:03.595 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec-profile', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}'] 2024-03-18T15:10:03.596 INFO:scrape: 2024-03-18T15:10:03.596 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f877106eba7] 2024-03-18T15:10:03.596 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f876fef2520] 2: pthread_kill() 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f877106eba7] 6: /usr/lib/ceph/libceph-common.so.2(+0x180d01) [0x7f877106ed01] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5577288a2851] 8: radosgw-admin(+0x8ab91a) [0x5577288aa91a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x5577288af9db] 10: main() 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f876fed9d90] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.596 INFO:scrape:1 jobs: ['7604938'] 2024-03-18T15:10:03.596 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec-profile', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}'] 2024-03-18T15:10:03.596 INFO:scrape: 2024-03-18T15:10:03.596 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f80f2780d4a] 2024-03-18T15:10:03.596 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib64/libc.so.6(+0x54db0) [0x7f80f1654db0] 2: /lib64/libc.so.6(+0xa154c) [0x7f80f16a154c] 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f80f2780d4a] 6: /usr/lib64/ceph/libceph-common.so.2(+0x180eae) [0x7f80f2780eae] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55f396750441] 8: radosgw-admin(+0x7cd44a) [0x55f39675844a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x55f39675cc5b] 10: main() 11: /lib64/libc.so.6(+0x3feb0) [0x7f80f163feb0] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.596 INFO:scrape:1 jobs: ['7604936'] 2024-03-18T15:10:03.596 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{bucket-check', 'validater/lockdep}', 'versioning}'] 2024-03-18T15:10:03.597 INFO:scrape: 2024-03-18T15:10:03.597 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f5a002a6ba7] 2024-03-18T15:10:03.597 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f59ff12a520] 2: pthread_kill() 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f5a002a6ba7] 6: /usr/lib/ceph/libceph-common.so.2(+0x180d01) [0x7f5a002a6d01] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a87de4f851] 8: radosgw-admin(+0x8ab91a) [0x55a87de5791a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x55a87de5c9db] 10: main() 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f59ff111d90] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.597 INFO:scrape:1 jobs: ['7604928'] 2024-03-18T15:10:03.597 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}'] 2024-03-18T15:10:03.598 INFO:scrape: 2024-03-18T15:10:03.598 INFO:scrape:Failure: failed to install new kernel version within timeout 2024-03-18T15:10:03.598 INFO:scrape:2 jobs: ['7604935', '7604937'] 2024-03-18T15:10:03.598 INFO:scrape:suites intersection: ['frontend/beast', 'ignore-pg-availability', 'objectstore/bluestore-bitmap', 'overrides', 'rgw/multifs/{clusters/fixed-2', 's3tests-branch', 'ubuntu_latest}'] 2024-03-18T15:10:03.598 INFO:scrape:suites union: ['frontend/beast', 'ignore-pg-availability', 'objectstore/bluestore-bitmap', 'overrides', 'rgw/multifs/{clusters/fixed-2', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 's3tests-branch', 'tasks/rgw_ragweed', 'tasks/rgw_s3tests', 'ubuntu_latest}'] 2024-03-18T15:10:03.598 INFO:scrape: 2024-03-18T15:10:03.598 INFO:scrape:Failure: Command failed (s3 tests against rgw) on smithi007 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-03-18T15:10:03.599 INFO:scrape:13 jobs: ['7604896', '7604883', '7604916', '7604872', '7604884', '7604876', '7604874', '7604897', '7604908', '7604922', '7604889', '7604934', '7604919'] 2024-03-18T15:10:03.599 INFO:scrape:suites intersection: ['ignore-pg-availability', 's3tests-branch'] 2024-03-18T15:10:03.599 INFO:scrape:suites union: ['1-ceph-install/install', '1-keycloak', '2-kms/barbican', '2-kms/testing', '2-kms/vault_kv', '2-kms/vault_old', '2-kms/vault_transit', '2-s3tests}}', '3-rgw/rgw', '4-tests/{s3tests}', 'clusters/fixed-1', 'frontend/beast', 'http', 'ignore-pg-availability', 'install', 'objectstore', 'objectstore/bluestore-bitmap', 'overrides', 'pool-type', 'rgw/crypt/{0-cluster/fixed-1', 'rgw/dbstore/{cluster', 'rgw/lifecycle/{cluster', 'rgw/multifs/{clusters/fixed-2', 'rgw/sts/{cluster', 'rgw/tempest/{0-install', 'rgw/thrash/{clusters/fixed-2', 'rgw/website/{clusters/fixed-2', 'rgw_frontend/beast', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-branch', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/rgw_s3tests', 'tasks/rgw_s3tests}', 'tasks/s3tests', 'tasks/s3tests-website', 'tasks/{0-install', 'thrasher/default', 'thrashosds-health', 'ubuntu_latest', 'ubuntu_latest}', 'workload/rgw_s3tests}'] 2024-03-18T15:10:03.599 INFO:scrape: 2024-03-18T15:10:03.600 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f87d64ccba7] 2024-03-18T15:10:03.600 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f87d5350520] 2: pthread_kill() 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f87d64ccba7] 6: /usr/lib/ceph/libceph-common.so.2(+0x180d01) [0x7f87d64ccd01] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5603cb067851] 8: radosgw-admin(+0x8ab91a) [0x5603cb06f91a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x5603cb0749db] 10: main() 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f87d5337d90] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.600 INFO:scrape:1 jobs: ['7604901'] 2024-03-18T15:10:03.600 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec-profile', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/lockdep}', 'versioning}'] 2024-03-18T15:10:03.600 INFO:scrape: 2024-03-18T15:10:03.600 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f41c2b62ba7] 2024-03-18T15:10:03.601 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f41c19e6520] 2: pthread_kill() 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f41c2b62ba7] 6: /usr/lib/ceph/libceph-common.so.2(+0x180d01) [0x7f41c2b62d01] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560c1c971851] 8: radosgw-admin(+0x8ab91a) [0x560c1c97991a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x560c1c97e9db] 10: main() 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f41c19cdd90] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.601 INFO:scrape:1 jobs: ['7604924'] 2024-03-18T15:10:03.601 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec-profile', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}'] 2024-03-18T15:10:03.601 INFO:scrape: 2024-03-18T15:10:03.602 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f7232b06ba7] 2024-03-18T15:10:03.602 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f723198a520] 2: pthread_kill() 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f7232b06ba7] 6: /usr/lib/ceph/libceph-common.so.2(+0x180d01) [0x7f7232b06d01] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x561c4ffbf851] 8: radosgw-admin(+0x8ab91a) [0x561c4ffc791a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x561c4ffcc9db] 10: main() 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f7231971d90] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.602 INFO:scrape:1 jobs: ['7604918'] 2024-03-18T15:10:03.602 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec-profile', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/lockdep}', 'versioning}'] 2024-03-18T15:10:03.602 INFO:scrape: 2024-03-18T15:10:03.602 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f82f1380d4a] 2024-03-18T15:10:03.602 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib64/libc.so.6(+0x54db0) [0x7f82f0254db0] 2: /lib64/libc.so.6(+0xa154c) [0x7f82f02a154c] 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f82f1380d4a] 6: /usr/lib64/ceph/libceph-common.so.2(+0x180eae) [0x7f82f1380eae] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x562a77b2d441] 8: radosgw-admin(+0x7cd44a) [0x562a77b3544a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x562a77b39c5b] 10: main() 11: /lib64/libc.so.6(+0x3feb0) [0x7f82f023feb0] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.603 INFO:scrape:1 jobs: ['7604941'] 2024-03-18T15:10:03.603 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}'] 2024-03-18T15:10:03.603 INFO:scrape: 2024-03-18T15:10:03.603 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f43f5f80d4a] 2024-03-18T15:10:03.603 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib64/libc.so.6(+0x54db0) [0x7f43f4e54db0] 2: /lib64/libc.so.6(+0xa154c) [0x7f43f4ea154c] 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f43f5f80d4a] 6: /usr/lib64/ceph/libceph-common.so.2(+0x180eae) [0x7f43f5f80eae] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5652b6117441] 8: radosgw-admin(+0x7cd44a) [0x5652b611f44a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x5652b6123c5b] 10: main() 11: /lib64/libc.so.6(+0x3feb0) [0x7f43f4e3feb0] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.603 INFO:scrape:1 jobs: ['7604895'] 2024-03-18T15:10:03.603 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{bucket-check', 'validater/lockdep}', 'versioning}'] 2024-03-18T15:10:03.603 INFO:scrape: 2024-03-18T15:10:03.603 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd9bfb80d4a] 2024-03-18T15:10:03.603 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib64/libc.so.6(+0x54db0) [0x7fd9bea54db0] 2: /lib64/libc.so.6(+0xa154c) [0x7fd9beaa154c] 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd9bfb80d4a] 6: /usr/lib64/ceph/libceph-common.so.2(+0x180eae) [0x7fd9bfb80eae] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d04159e441] 8: radosgw-admin(+0x7cd44a) [0x55d0415a644a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x55d0415aac5b] 10: main() 11: /lib64/libc.so.6(+0x3feb0) [0x7fd9bea3feb0] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.604 INFO:scrape:1 jobs: ['7604903'] 2024-03-18T15:10:03.604 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}'] 2024-03-18T15:10:03.604 INFO:scrape: 2024-03-18T15:10:03.604 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f7eb4d5aba7] 2024-03-18T15:10:03.604 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f7eb3bde520] 2: pthread_kill() 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f7eb4d5aba7] 6: /usr/lib/ceph/libceph-common.so.2(+0x180d01) [0x7f7eb4d5ad01] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55bac107a851] 8: radosgw-admin(+0x8ab91a) [0x55bac108291a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x55bac10879db] 10: main() 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f7eb3bc5d90] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.604 INFO:scrape:1 jobs: ['7604927'] 2024-03-18T15:10:03.604 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/lockdep}', 'versioning}'] 2024-03-18T15:10:03.604 INFO:scrape: 2024-03-18T15:10:03.604 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fa148380d4a] 2024-03-18T15:10:03.604 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib64/libc.so.6(+0x54db0) [0x7fa147254db0] 2: /lib64/libc.so.6(+0xa154c) [0x7fa1472a154c] 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fa148380d4a] 6: /usr/lib64/ceph/libceph-common.so.2(+0x180eae) [0x7fa148380eae] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a97b9d9441] 8: radosgw-admin(+0x7cd44a) [0x55a97b9e144a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x55a97b9e5c5b] 10: main() 11: /lib64/libc.so.6(+0x3feb0) [0x7fa14723feb0] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.605 INFO:scrape:1 jobs: ['7604906'] 2024-03-18T15:10:03.605 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{bucket-check', 'validater/lockdep}', 'versioning}'] 2024-03-18T15:10:03.605 INFO:scrape: 2024-03-18T15:10:03.605 INFO:scrape:Dead: None 2024-03-18T15:10:03.605 INFO:scrape:1 jobs: ['7604921'] 2024-03-18T15:10:03.605 INFO:scrape: 2024-03-18T15:10:03.605 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7faf9ab80d4a] 2024-03-18T15:10:03.605 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib64/libc.so.6(+0x54db0) [0x7faf99a54db0] 2: /lib64/libc.so.6(+0xa154c) [0x7faf99aa154c] 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7faf9ab80d4a] 6: /usr/lib64/ceph/libceph-common.so.2(+0x180eae) [0x7faf9ab80eae] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e3c48c2441] 8: radosgw-admin(+0x7cd44a) [0x55e3c48ca44a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x55e3c48cec5b] 10: main() 11: /lib64/libc.so.6(+0x3feb0) [0x7faf99a3feb0] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.605 INFO:scrape:1 jobs: ['7604890'] 2024-03-18T15:10:03.605 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}'] 2024-03-18T15:10:03.606 INFO:scrape: 2024-03-18T15:10:03.606 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f37e002aba7] 2024-03-18T15:10:03.606 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f37deeae520] 2: pthread_kill() 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f37e002aba7] 6: /usr/lib/ceph/libceph-common.so.2(+0x180d01) [0x7f37e002ad01] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55bcd41d7851] 8: radosgw-admin(+0x8ab91a) [0x55bcd41df91a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x55bcd41e49db] 10: main() 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f37dee95d90] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.606 INFO:scrape:1 jobs: ['7604911'] 2024-03-18T15:10:03.606 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}'] 2024-03-18T15:10:03.606 INFO:scrape: 2024-03-18T15:10:03.606 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fb03a9c5ba7] 2024-03-18T15:10:03.606 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fb039849520] 2: pthread_kill() 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fb03a9c5ba7] 6: /usr/lib/ceph/libceph-common.so.2(+0x180d01) [0x7fb03a9c5d01] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5593a8d74851] 8: radosgw-admin(+0x8ab91a) [0x5593a8d7c91a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x5593a8d819db] 10: main() 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fb039830d90] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.606 INFO:scrape:1 jobs: ['7604920'] 2024-03-18T15:10:03.606 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}'] 2024-03-18T15:10:03.607 INFO:scrape: 2024-03-18T15:10:03.607 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f7007780d4a] 2024-03-18T15:10:03.607 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib64/libc.so.6(+0x54db0) [0x7f7006654db0] 2: /lib64/libc.so.6(+0xa154c) [0x7f70066a154c] 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f7007780d4a] 6: /usr/lib64/ceph/libceph-common.so.2(+0x180eae) [0x7f7007780eae] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55a41f1f3441] 8: radosgw-admin(+0x7cd44a) [0x55a41f1fb44a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x55a41f1ffc5b] 10: main() 11: /lib64/libc.so.6(+0x3feb0) [0x7f700663feb0] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.607 INFO:scrape:1 jobs: ['7604892'] 2024-03-18T15:10:03.607 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{bucket-check', 'validater/lockdep}', 'versioning}'] 2024-03-18T15:10:03.607 INFO:scrape: 2024-03-18T15:10:03.607 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f26e6d1eba7] 2024-03-18T15:10:03.607 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f26e5ba2520] 2: pthread_kill() 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f26e6d1eba7] 6: /usr/lib/ceph/libceph-common.so.2(+0x180d01) [0x7f26e6d1ed01] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5588170e3851] 8: radosgw-admin(+0x8ab91a) [0x5588170eb91a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x5588170f09db] 10: main() 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f26e5b89d90] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.607 INFO:scrape:1 jobs: ['7604932'] 2024-03-18T15:10:03.607 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec-profile', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/lockdep}', 'versioning}'] 2024-03-18T15:10:03.608 INFO:scrape: 2024-03-18T15:10:03.608 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f5bf3980d4a] 2024-03-18T15:10:03.608 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib64/libc.so.6(+0x54db0) [0x7f5bf2654db0] 2: /lib64/libc.so.6(+0xa154c) [0x7f5bf26a154c] 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f5bf3980d4a] 6: /usr/lib64/ceph/libceph-common.so.2(+0x180eae) [0x7f5bf3980eae] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x561cdb7ef441] 8: radosgw-admin(+0x7cd44a) [0x561cdb7f744a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x561cdb7fbc5b] 10: main() 11: /lib64/libc.so.6(+0x3feb0) [0x7f5bf263feb0] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.608 INFO:scrape:1 jobs: ['7604910'] 2024-03-18T15:10:03.608 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{bucket-check', 'validater/lockdep}', 'versioning}'] 2024-03-18T15:10:03.608 INFO:scrape: 2024-03-18T15:10:03.608 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f8783980d4a] 2024-03-18T15:10:03.608 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib64/libc.so.6(+0x54db0) [0x7f8782854db0] 2: /lib64/libc.so.6(+0xa154c) [0x7f87828a154c] 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f8783980d4a] 6: /usr/lib64/ceph/libceph-common.so.2(+0x180eae) [0x7f8783980eae] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x563c28468441] 8: radosgw-admin(+0x7cd44a) [0x563c2847044a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x563c28474c5b] 10: main() 11: /lib64/libc.so.6(+0x3feb0) [0x7f878283feb0] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.608 INFO:scrape:1 jobs: ['7604933'] 2024-03-18T15:10:03.608 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}'] 2024-03-18T15:10:03.608 INFO:scrape: 2024-03-18T15:10:03.609 INFO:scrape:Failure: Command failed (workunit test rgw/run-bucket-check.sh) on smithi052 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9e0ae027f7eb299a29cfc01325e109bb45e4fbf5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh' 2024-03-18T15:10:03.609 INFO:scrape:1 jobs: ['7604893'] 2024-03-18T15:10:03.609 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec-profile', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}'] 2024-03-18T15:10:03.609 INFO:scrape: 2024-03-18T15:10:03.609 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f5e05da2ba7] 2024-03-18T15:10:03.609 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f5e04c26520] 2: pthread_kill() 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f5e05da2ba7] 6: /usr/lib/ceph/libceph-common.so.2(+0x180d01) [0x7f5e05da2d01] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557efde7a851] 8: radosgw-admin(+0x8ab91a) [0x557efde8291a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x557efde879db] 10: main() 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f5e04c0dd90] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.609 INFO:scrape:1 jobs: ['7604939'] 2024-03-18T15:10:03.609 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/lockdep}', 'versioning}'] 2024-03-18T15:10:03.609 INFO:scrape: 2024-03-18T15:10:03.609 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fb3b8c9bba7] 2024-03-18T15:10:03.610 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fb3b7b1f520] 2: pthread_kill() 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fb3b8c9bba7] 6: /usr/lib/ceph/libceph-common.so.2(+0x180d01) [0x7fb3b8c9bd01] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x56340db43851] 8: radosgw-admin(+0x8ab91a) [0x56340db4b91a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x56340db509db] 10: main() 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fb3b7b06d90] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.610 INFO:scrape:1 jobs: ['7604888'] 2024-03-18T15:10:03.610 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec-profile', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/lockdep}', 'versioning}'] 2024-03-18T15:10:03.610 INFO:scrape: 2024-03-18T15:10:03.610 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f4772f80d4a] 2024-03-18T15:10:03.610 INFO:scrape:ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 1: /lib64/libc.so.6(+0x54db0) [0x7f4771e54db0] 2: /lib64/libc.so.6(+0xa154c) [0x7f4771ea154c] 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f4772f80d4a] 6: /usr/lib64/ceph/libceph-common.so.2(+0x180eae) [0x7f4772f80eae] 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x5573983af441] 8: radosgw-admin(+0x7cd44a) [0x5573983b744a] 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x5573983bbc5b] 10: main() 11: /lib64/libc.so.6(+0x3feb0) [0x7f4771e3feb0] 12: __libc_start_main() 13: _start() 2024-03-18T15:10:03.610 INFO:scrape:1 jobs: ['7604898'] 2024-03-18T15:10:03.610 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}'] 2024-03-18T15:10:03.610 INFO:scrape: 2024-03-18T15:10:03.614 INFO:teuthology.results:Sending results to ceph-qa@ceph.com: Test Run: teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/ failed: 40 dead: 1 running: 0 waiting: 0 queued: 0 passed: 30 Fail ================================================================= [7604872] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest} ----------------------------------------------------------------- time: 00:30:58 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604872/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604872/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3a9b6a29fede4de180a65293df8b8347 Command failed (s3 tests against rgw) on smithi019 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604874] rgw/dbstore/{cluster ignore-pg-availability overrides s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/rgw_s3tests} ----------------------------------------------------------------- time: 00:17:20 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604874/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604874/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bfc200bf300849a28f4fec472b98f463 Command failed (s3 tests against rgw) on smithi111 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3 and not fails_on_dbstore'" [7604876] rgw/lifecycle/{cluster ignore-pg-availability overrides s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/rgw_s3tests} ----------------------------------------------------------------- time: 00:39:23 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604876/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604876/ sentry: https://sentry.ceph.com/organizations/ceph/?query=893b9c291fdc47c7bcfd6ea522f9ed69 Command failed (s3 tests against rgw) on smithi151 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604879] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zones supported-random-distro$/{ubuntu_latest} tasks/test_multi} ----------------------------------------------------------------- time: 02:56:49 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604879/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604879/ sentry: https://sentry.ceph.com/organizations/ceph/?query=751b224c356949869d40aead09783059 rgw multisite test failures [7604883] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast s3tests-branch supported-random-distro$/{centos_latest} tasks/{0-install 1-keycloak 2-s3tests}} ----------------------------------------------------------------- time: 00:33:33 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604883/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604883/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a247b40d0b884aa09e9adff8053ab4b6 Command failed (s3 tests against rgw) on smithi006 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not fails_with_subdomain and not sse_s3'" [7604884] rgw/tempest/{0-install clusters/fixed-1 frontend/beast ignore-pg-availability overrides s3tests-branch tasks/s3tests ubuntu_latest} ----------------------------------------------------------------- time: 00:33:58 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604884/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604884/ sentry: https://sentry.ceph.com/organizations/ceph/?query=91da78019cbb41a7bda9bb2a0a643fa0 Command failed (s3 tests against rgw) on smithi032 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604888] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} ----------------------------------------------------------------- time: 00:41:21 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604888/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604888/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5745eac234b6451f892e286b4446ee1b Command failed (s3 tests against rgw) on smithi088 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604889] rgw/website/{clusters/fixed-2 frontend/beast http ignore-pg-availability overrides s3tests-branch tasks/s3tests-website ubuntu_latest} ----------------------------------------------------------------- time: 00:21:58 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604889/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604889/ sentry: https://sentry.ceph.com/organizations/ceph/?query=44c6a10cc05e43d89779c8a4ba306c8b Command failed (s3 tests against rgw) on smithi005 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604890] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} ----------------------------------------------------------------- time: 01:05:04 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604890/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604890/ sentry: https://sentry.ceph.com/organizations/ceph/?query=99b2cc9c384443af8b375575de23a6b3 Command failed (s3 tests against rgw) on smithi078 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604892] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} ----------------------------------------------------------------- time: 00:32:20 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604892/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604892/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f326756ba64f48649b528d3c027e9434 Command failed (s3 tests against rgw) on smithi156 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604893] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} ----------------------------------------------------------------- time: 00:15:48 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604893/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604893/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ed21ceb473d9408cba55cca8972b6d3e Command failed (workunit test rgw/run-bucket-check.sh) on smithi052 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9e0ae027f7eb299a29cfc01325e109bb45e4fbf5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh' [7604895] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} ----------------------------------------------------------------- time: 00:32:36 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604895/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604895/ sentry: https://sentry.ceph.com/organizations/ceph/?query=319aa31aff9445078a6f4424a469a59c Command failed (s3 tests against rgw) on smithi129 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604896] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest} ----------------------------------------------------------------- time: 00:21:25 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604896/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604896/ sentry: https://sentry.ceph.com/organizations/ceph/?query=06e1060d0a9644aaa1d04a5eb2e9a1d5 Command failed (s3 tests against rgw) on smithi007 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604897] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_s3tests ubuntu_latest} ----------------------------------------------------------------- time: 00:21:39 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604897/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604897/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8b37cd565648471b9f898c5f05a4b104 Command failed (s3 tests against rgw) on smithi059 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604898] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} ----------------------------------------------------------------- time: 01:04:55 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604898/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604898/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3a1b0b2e76a043f9873d0ee9c152bc9d Command failed (s3 tests against rgw) on smithi172 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604901] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} ----------------------------------------------------------------- time: 00:44:34 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604901/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604901/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fe8bae6915824879978d3a8a7994b623 Command failed (s3 tests against rgw) on smithi097 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604903] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} ----------------------------------------------------------------- time: 00:53:14 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604903/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604903/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3950a13fa3094c11b5e55e8fd2669856 Command failed (s3 tests against rgw) on smithi055 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604906] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} ----------------------------------------------------------------- time: 00:35:39 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604906/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604906/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7c035bc8c8f94e86b98b22937a072ea2 Command failed (s3 tests against rgw) on smithi033 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604907] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} ----------------------------------------------------------------- time: 00:52:24 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604907/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604907/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bf6312aa2c384691bd01f456801b92bb Command failed (s3 tests against rgw) on smithi064 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604908] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest} ----------------------------------------------------------------- time: 00:21:14 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604908/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604908/ sentry: https://sentry.ceph.com/organizations/ceph/?query=efcfed9ad2ec41e4a67b154988dab1e9 Command failed (s3 tests against rgw) on smithi096 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604910] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} ----------------------------------------------------------------- time: 00:35:34 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604910/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604910/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b50184dc0cfd488ba9eb3a00001cbe9a Command failed (s3 tests against rgw) on smithi123 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604911] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} ----------------------------------------------------------------- time: 01:18:16 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604911/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604911/ sentry: https://sentry.ceph.com/organizations/ceph/?query=70f5e64aba314099bfd834879cf3eda5 Command failed (s3 tests against rgw) on smithi046 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604914] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{centos_latest} tasks/test_multi} ----------------------------------------------------------------- time: 02:47:26 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604914/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604914/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9923d7c2490c4d9ebf25660fd3bd4ce2 rgw multisite test failures [7604916] rgw/thrash/{clusters/fixed-2 frontend/beast ignore-pg-availability install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} ----------------------------------------------------------------- time: 00:22:29 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604916/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604916/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e41298604b2d435a8fb3b25e108d9280 Command failed (s3 tests against rgw) on smithi017 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604918] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} ----------------------------------------------------------------- time: 00:43:09 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604918/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604918/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2b87af478edf4636a3ede52bd3c4c2d0 Command failed (s3 tests against rgw) on smithi002 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604919] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_s3tests ubuntu_latest} ----------------------------------------------------------------- time: 00:21:40 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604919/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604919/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b7c24287d01a419eb1f1429bfdb4ebfa Command failed (s3 tests against rgw) on smithi001 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604920] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} ----------------------------------------------------------------- time: 01:52:41 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604920/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604920/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a6df59cd03a541a3b7463c73815a1dde Command failed (s3 tests against rgw) on smithi066 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604922] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest} ----------------------------------------------------------------- time: 00:21:06 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604922/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604922/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d018642a00ed42f4ad711dd77d563825 Command failed (s3 tests against rgw) on smithi122 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604924] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} ----------------------------------------------------------------- time: 01:47:30 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604924/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604924/ sentry: https://sentry.ceph.com/organizations/ceph/?query=452e8469c60141808460b625bf0c66ae Command failed (s3 tests against rgw) on smithi146 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604927] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} ----------------------------------------------------------------- time: 00:42:00 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604927/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604927/ sentry: https://sentry.ceph.com/organizations/ceph/?query=615fd65a22ae49f49f81af1837fa1c84 Command failed (s3 tests against rgw) on smithi060 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604928] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} ----------------------------------------------------------------- time: 01:49:26 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604928/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604928/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5103cb41a03a4ef8b6a29c091a2163d9 Command failed (s3 tests against rgw) on smithi012 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604932] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} ----------------------------------------------------------------- time: 00:45:31 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604932/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604932/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3013a4e246d249888d227dae3f7dd042 Command failed (s3 tests against rgw) on smithi023 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604933] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} ----------------------------------------------------------------- time: 00:51:10 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604933/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604933/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a5a9d50f549c4260b0041edbfb68a60f Command failed (s3 tests against rgw) on smithi040 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604934] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest} ----------------------------------------------------------------- time: 00:22:24 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604934/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604934/ sentry: https://sentry.ceph.com/organizations/ceph/?query=01fffb5643dd4808bedcb6b2c4f99ec9 Command failed (s3 tests against rgw) on smithi042 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain'" [7604935] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_ragweed ubuntu_latest} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604935/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604935/ sentry: https://sentry.ceph.com/organizations/ceph/?query=56b1f584a11146fb95909590c63445a0 failed to install new kernel version within timeout [7604936] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} ----------------------------------------------------------------- time: 00:36:37 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604936/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604936/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e93807d8d2f54c39af43d588cd6f1dcb Command failed (s3 tests against rgw) on smithi063 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604937] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_s3tests ubuntu_latest} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604937/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604937/ sentry: https://sentry.ceph.com/organizations/ceph/?query=15a074d1e44a41488665afdebfce1ac6 failed to install new kernel version within timeout [7604938] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} ----------------------------------------------------------------- time: 01:14:29 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604938/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604938/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d43424275dda459ea97cd9209614d5d7 Command failed (s3 tests against rgw) on smithi102 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604939] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} ----------------------------------------------------------------- time: 00:45:35 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604939/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604939/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e318d2edb982486eb79ff7cfc0323c0d Command failed (s3 tests against rgw) on smithi090 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" [7604941] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} ----------------------------------------------------------------- time: 00:47:57 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604941/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604941/ sentry: https://sentry.ceph.com/organizations/ceph/?query=52bd7311b2954c2882dcf9524383e63a Command failed (s3 tests against rgw) on smithi086 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" Dead ================================================================= [7604921] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604921/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604921/ Error reimaging machines: Failed to power on smithi103 Pass ================================================================= [7604871] rgw/cloud-transition/{cluster ignore-pg-availability overrides s3tests-branch supported-random-distro$/{centos_latest} tasks/cloud_transition_s3tests} time: 00:26:21 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604871/ [7604873] rgw/d4n/{cluster ignore-pg-availability overrides supported-random-distro$/{centos_latest} tasks/rgw_d4ntests} time: 00:10:08 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604873/ [7604875] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default ignore-pg-availability overrides s3a-hadoop supported-random-distro$/{ubuntu_latest}} time: 00:19:58 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604875/ [7604877] rgw/lua/{beast bluestore-bitmap fixed-2 overrides supported-distros/{centos_latest} tasks/{0-install test_lua}} time: 00:11:56 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604877/ [7604878] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_bucket_quota ubuntu_latest} time: 00:15:01 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604878/ [7604880] rgw/notifications/{beast bluestore-bitmap centos_latest fixed-2 ignore-pg-availability overrides tasks/{0-install test_amqp test_kafka test_others}} time: 00:44:25 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604880/ [7604881] rgw/service-token/{clusters/fixed-1 frontend/beast ignore-pg-availability overrides tasks/service-token ubuntu_latest} time: 00:13:32 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604881/ [7604882] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_latest}} time: 00:13:05 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604882/ [7604885] rgw/thrash/{clusters/fixed-2 frontend/beast ignore-pg-availability install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} time: 00:16:09 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604885/ [7604886] rgw/tools/{centos_latest cluster ignore-pg-availability tasks} time: 02:37:09 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604886/ [7604887] rgw/upgrade/{1-install/quincy/{distro$/{centos_latest} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides} time: 00:14:59 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604887/ [7604891] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_multipart_upload ubuntu_latest} time: 00:13:49 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604891/ [7604894] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_ragweed ubuntu_latest} time: 00:14:49 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604894/ [7604899] rgw/thrash/{clusters/fixed-2 frontend/beast ignore-pg-availability install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} time: 00:14:01 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604899/ [7604900] rgw/upgrade/{1-install/reef/{distro$/{ubuntu_latest} install overrides} 2-setup 3-upgrade-sequence/rgws-then-osds cluster frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides} time: 00:15:38 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604900/ [7604902] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_user_quota ubuntu_latest} time: 00:13:34 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604902/ [7604904] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_bucket_quota ubuntu_latest} time: 00:15:10 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604904/ [7604905] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{ubuntu_latest}} time: 00:14:49 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604905/ [7604909] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_multipart_upload ubuntu_latest} time: 00:13:55 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604909/ [7604912] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_ragweed ubuntu_latest} time: 00:14:58 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604912/ [7604913] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 ignore-pg-availability overrides s3a-hadoop supported-random-distro$/{ubuntu_latest}} time: 00:18:20 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604913/ [7604915] rgw/tempest/{0-install clusters/fixed-1 frontend/beast ignore-pg-availability overrides s3tests-branch tasks/tempest ubuntu_latest} time: 00:21:12 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604915/ [7604917] rgw/upgrade/{1-install/reef/{distro$/{centos_latest} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides} time: 00:15:00 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604917/ [7604923] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_user_quota ubuntu_latest} time: 00:13:32 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604923/ [7604925] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_bucket_quota ubuntu_latest} time: 00:16:31 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604925/ [7604926] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} time: 00:15:18 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604926/ [7604929] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_multipart_upload ubuntu_latest} time: 00:13:56 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604929/ [7604930] rgw/thrash/{clusters/fixed-2 frontend/beast ignore-pg-availability install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} time: 00:14:10 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604930/ [7604931] rgw/upgrade/{1-install/quincy/{distro$/{centos_latest} install overrides} 2-setup 3-upgrade-sequence/rgws-then-osds cluster frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides} time: 00:15:14 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604931/ [7604940] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_user_quota ubuntu_latest} time: 00:13:50 info: https://pulpito.ceph.com/teuthology-2024-03-15_21:40:02-rgw-squid-distro-default-smithi/7604940/