2023-09-18T14:17:52.623 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-09-18T14:17:52.624 INFO:teuthology.results:seed: '1266'
2023-09-18T15:24:43.946 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-09-18T15:24:43.950 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-09-18T17:17:52.295 INFO:teuthology.results:Tests finished! gathering results...
2023-09-18T17:17:56.796 INFO:scrape:Found 70 jobs
2023-09-18T17:17:58.087 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397641/teuthology.log
2023-09-18T17:17:58.088 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397641/teuthology.log
2023-09-18T17:17:58.089 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397641/teuthology.log
2023-09-18T17:18:00.061 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397637/teuthology.log matches
2023-09-18T17:18:00.062 WARNING:scrape:Not-understood crash indication 
2023-09-18T17:18:00.109 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397637/teuthology.log matches
2023-09-18T17:18:00.110 WARNING:scrape:Not-understood crash indication 
2023-09-18T17:18:00.168 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397637/teuthology.log matches
2023-09-18T17:18:00.169 WARNING:scrape:Not-understood crash indication 
2023-09-18T17:18:00.215 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397637/teuthology.log matches
2023-09-18T17:18:00.215 WARNING:scrape:Not-understood crash indication 
2023-09-18T17:18:00.262 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397637/teuthology.log matches
2023-09-18T17:18:00.263 WARNING:scrape:Not-understood crash indication 
2023-09-18T17:18:00.309 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397637/teuthology.log matches
2023-09-18T17:18:00.310 WARNING:scrape:Not-understood crash indication 
2023-09-18T17:18:01.215 WARNING:scrape:Not-understood crash indication 2023-09-18T16:17:53.428 INFO:tasks.rgw.c1.client.1.smithi084.stderr:daemon-helper: command crashed with signal 11
2023-09-18T17:18:01.216 WARNING:scrape:Not-understood crash indication 2023-09-18T16:58:06.497 INFO:tasks.rgw_multisite_tests:tasks.rgw.c1.client.1.smithi084.stderr: INFO: daemon-helper: command crashed with signal 11
2023-09-18T17:18:01.217 WARNING:scrape:Not-understood crash indication 
2023-09-18T17:18:01.591 WARNING:scrape:Not-understood crash indication 2023-09-18T16:17:53.428 INFO:tasks.rgw.c1.client.1.smithi084.stderr:daemon-helper: command crashed with signal 11
2023-09-18T17:18:01.592 WARNING:scrape:Not-understood crash indication 2023-09-18T16:58:06.497 INFO:tasks.rgw_multisite_tests:tasks.rgw.c1.client.1.smithi084.stderr: INFO: daemon-helper: command crashed with signal 11
2023-09-18T17:18:01.592 WARNING:scrape:Not-understood crash indication 
2023-09-18T17:18:01.994 WARNING:scrape:Not-understood crash indication 2023-09-18T16:17:53.428 INFO:tasks.rgw.c1.client.1.smithi084.stderr:daemon-helper: command crashed with signal 11
2023-09-18T17:18:01.995 WARNING:scrape:Not-understood crash indication 2023-09-18T16:58:06.497 INFO:tasks.rgw_multisite_tests:tasks.rgw.c1.client.1.smithi084.stderr: INFO: daemon-helper: command crashed with signal 11
2023-09-18T17:18:01.996 WARNING:scrape:Not-understood crash indication 
2023-09-18T17:18:02.395 WARNING:scrape:Not-understood crash indication 2023-09-18T16:17:53.428 INFO:tasks.rgw.c1.client.1.smithi084.stderr:daemon-helper: command crashed with signal 11
2023-09-18T17:18:02.396 WARNING:scrape:Not-understood crash indication 2023-09-18T16:58:06.497 INFO:tasks.rgw_multisite_tests:tasks.rgw.c1.client.1.smithi084.stderr: INFO: daemon-helper: command crashed with signal 11
2023-09-18T17:18:02.396 WARNING:scrape:Not-understood crash indication 
2023-09-18T17:18:02.786 WARNING:scrape:Not-understood crash indication 2023-09-18T16:17:53.428 INFO:tasks.rgw.c1.client.1.smithi084.stderr:daemon-helper: command crashed with signal 11
2023-09-18T17:18:02.786 WARNING:scrape:Not-understood crash indication 2023-09-18T16:58:06.497 INFO:tasks.rgw_multisite_tests:tasks.rgw.c1.client.1.smithi084.stderr: INFO: daemon-helper: command crashed with signal 11
2023-09-18T17:18:02.787 WARNING:scrape:Not-understood crash indication 
2023-09-18T17:18:03.184 WARNING:scrape:Not-understood crash indication 2023-09-18T16:17:53.428 INFO:tasks.rgw.c1.client.1.smithi084.stderr:daemon-helper: command crashed with signal 11
2023-09-18T17:18:03.185 WARNING:scrape:Not-understood crash indication 2023-09-18T16:58:06.497 INFO:tasks.rgw_multisite_tests:tasks.rgw.c1.client.1.smithi084.stderr: INFO: daemon-helper: command crashed with signal 11
2023-09-18T17:18:03.185 WARNING:scrape:Not-understood crash indication 
2023-09-18T17:18:03.579 WARNING:scrape:Not-understood crash indication 2023-09-18T16:17:53.428 INFO:tasks.rgw.c1.client.1.smithi084.stderr:daemon-helper: command crashed with signal 11
2023-09-18T17:18:03.580 WARNING:scrape:Not-understood crash indication 2023-09-18T16:58:06.497 INFO:tasks.rgw_multisite_tests:tasks.rgw.c1.client.1.smithi084.stderr: INFO: daemon-helper: command crashed with signal 11
2023-09-18T17:18:03.580 WARNING:scrape:Not-understood crash indication 
2023-09-18T17:18:03.966 WARNING:scrape:Not-understood crash indication 2023-09-18T16:17:53.428 INFO:tasks.rgw.c1.client.1.smithi084.stderr:daemon-helper: command crashed with signal 11
2023-09-18T17:18:03.967 WARNING:scrape:Not-understood crash indication 2023-09-18T16:58:06.497 INFO:tasks.rgw_multisite_tests:tasks.rgw.c1.client.1.smithi084.stderr: INFO: daemon-helper: command crashed with signal 11
2023-09-18T17:18:03.967 WARNING:scrape:Not-understood crash indication 
2023-09-18T17:18:04.366 WARNING:scrape:Not-understood crash indication 2023-09-18T16:17:53.428 INFO:tasks.rgw.c1.client.1.smithi084.stderr:daemon-helper: command crashed with signal 11
2023-09-18T17:18:04.367 WARNING:scrape:Not-understood crash indication 2023-09-18T16:58:06.497 INFO:tasks.rgw_multisite_tests:tasks.rgw.c1.client.1.smithi084.stderr: INFO: daemon-helper: command crashed with signal 11
2023-09-18T17:18:04.368 WARNING:scrape:Not-understood crash indication 
2023-09-18T17:18:04.810 INFO:scrape:Found 9 distinct failure reasons
2023-09-18T17:18:04.811 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fa1ce963155]
2023-09-18T17:18:04.812 INFO:scrape:ceph version 18.0.0-5652-g74e83480 (74e834800f96af4850dac39e00acc2e6c3080eb1) reef (dev)
 1: /lib64/libc.so.6(+0x54db0) [0x7fa1cbe54db0]
 2: /lib64/libc.so.6(+0xa154c) [0x7fa1cbea154c]
 3: raise()
 4: abort()
 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fa1ce963155]
 6: /usr/lib64/ceph/libceph-common.so.2(+0x1632b9) [0x7fa1ce9632b9]
 7: (std::__detail::__variant::__gen_vtable_impl<std::__detail::__variant::_Multi_array<std::__detail::__variant::__deduce_visit_result<int> (*)(FaultInjector<std::basic_string_view<char, std::char_traits<char> > >::check(std::basic_string_view<char, std::char_traits<char> > const&) const::visitor&&, std::variant<std::monostate, InjectAbort, InjectError, InjectDelay> const&)>, std::integer_sequence<unsigned long, 1ul> >::__visit_invoke(FaultInjector<std::basic_string_view<char, std::char_traits<char> > >::check(std::basic_string_view<char, std::char_traits<char> > const&) const::visitor&&, std::variant<std::monostate, InjectAbort, InjectError, InjectDelay> const&)+0x41) [0x55b625694a21]
 8: radosgw-admin(+0x7725fa) [0x55b6256a25fa]
 9: (RGWBucketReshard::execute(int, FaultInjector<std::basic_string_view<char, std::char_traits<char> > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x55b6256aac5b]
 10: main()
 11: /lib64/libc.so.6(+0x3feb0) [0x7fa1cbe3feb0]
 12: __libc_start_main()
 13: _start()
2023-09-18T17:18:04.812 INFO:scrape:1 jobs: ['7397660']
2023-09-18T17:18:04.812 INFO:scrape:suites: ['clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{cls', 'validater/valgrind}', 'versioning}']
2023-09-18T17:18:04.813 INFO:scrape:
2023-09-18T17:18:04.813 INFO:scrape:Failure: Command failed on smithi029 with status 100: 'DEBIAN_FRONTEND=noninteractive sudo -E apt-get -y --force-yes install python-dev'
2023-09-18T17:18:04.814 INFO:scrape:1 jobs: ['7397657']
2023-09-18T17:18:04.814 INFO:scrape:suites: ['1-ceph-install/install', '2-kms/kmip', '3-rgw/rgw', '4-tests/{s3tests}', 'ignore-pg-availability', 'rgw/crypt/{0-cluster/fixed-1', 's3tests-branch', 'ubuntu_latest}']
2023-09-18T17:18:04.815 INFO:scrape:
2023-09-18T17:18:04.815 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7efc91563155]
2023-09-18T17:18:04.816 INFO:scrape:ceph version 18.0.0-5652-g74e83480 (74e834800f96af4850dac39e00acc2e6c3080eb1) reef (dev)
 1: /lib64/libc.so.6(+0x54db0) [0x7efc8ea54db0]
 2: /lib64/libc.so.6(+0xa154c) [0x7efc8eaa154c]
 3: raise()
 4: abort()
 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7efc91563155]
 6: /usr/lib64/ceph/libceph-common.so.2(+0x1632b9) [0x7efc915632b9]
 7: (std::__detail::__variant::__gen_vtable_impl<std::__detail::__variant::_Multi_array<std::__detail::__variant::__deduce_visit_result<int> (*)(FaultInjector<std::basic_string_view<char, std::char_traits<char> > >::check(std::basic_string_view<char, std::char_traits<char> > const&) const::visitor&&, std::variant<std::monostate, InjectAbort, InjectError, InjectDelay> const&)>, std::integer_sequence<unsigned long, 1ul> >::__visit_invoke(FaultInjector<std::basic_string_view<char, std::char_traits<char> > >::check(std::basic_string_view<char, std::char_traits<char> > const&) const::visitor&&, std::variant<std::monostate, InjectAbort, InjectError, InjectDelay> const&)+0x41) [0x563b7464ea21]
 8: radosgw-admin(+0x7725fa) [0x563b7465c5fa]
 9: (RGWBucketReshard::execute(int, FaultInjector<std::basic_string_view<char, std::char_traits<char> > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x563b74664c5b]
 10: main()
 11: /lib64/libc.so.6(+0x3feb0) [0x7efc8ea3feb0]
 12: __libc_start_main()
 13: _start()
2023-09-18T17:18:04.816 INFO:scrape:1 jobs: ['7397655']
2023-09-18T17:18:04.817 INFO:scrape:suites: ['clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec-profile', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{cls', 'validater/valgrind}', 'versioning}']
2023-09-18T17:18:04.817 INFO:scrape:
2023-09-18T17:18:04.817 INFO:scrape:Failure: Command failed on smithi071 with status 127: 'cd /home/ubuntu/cephtest/keycloak-11.0.0 && cd bin && ./jboss-cli.sh --file=adapter-elytron-install-offline.cli'
2023-09-18T17:18:04.818 INFO:scrape:1 jobs: ['7397645']
2023-09-18T17:18:04.818 INFO:scrape:suites: ['1-keycloak', '2-s3tests}}', 'ignore-pg-availability', 'objectstore', 'overrides', 'pool-type', 'rgw/sts/{cluster', 'rgw_frontend/beast', 's3tests-branch', 'supported-random-distro$/{centos_latest}', 'tasks/{0-install']
2023-09-18T17:18:04.819 INFO:scrape:
2023-09-18T17:18:04.819 INFO:scrape:Dead: None
2023-09-18T17:18:04.820 INFO:scrape:1 jobs: ['7397641']
2023-09-18T17:18:04.820 INFO:scrape:
2023-09-18T17:18:04.820 INFO:scrape:Failure: Command failed (bucket notification tests against different endpoints) on smithi125 with status 1: "BNTESTS_CONF=/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/bn-tests.client.0.conf /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/bin/python -m nose -s /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py -v -a '!kafka_test,!amqp_test,!amqp_ssl_test,!kafka_security_test,!modification_required,!manual_test'"
2023-09-18T17:18:04.821 INFO:scrape:1 jobs: ['7397642']
2023-09-18T17:18:04.821 INFO:scrape:suites: ['bluestore-bitmap', 'fixed-2', 'ignore-pg-availability', 'overrides', 'rgw/notifications/{beast', 'supported-all-distro$/{centos_8}', 'tasks/{0-install', 'test_amqp', 'test_kafka', 'test_others}}']
2023-09-18T17:18:04.822 INFO:scrape:
2023-09-18T17:18:04.822 INFO:scrape:Failure: Command failed (s3 tests against rgw) on smithi031 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3 and cloud_transition'"
2023-09-18T17:18:04.822 INFO:scrape:2 jobs: ['7397634', '7397637']
2023-09-18T17:18:04.823 INFO:scrape:suites intersection: ['ignore-pg-availability', 'overrides', 's3tests-branch', 'supported-random-distro$/{centos_latest}']
2023-09-18T17:18:04.823 INFO:scrape:suites union: ['ignore-pg-availability', 'overrides', 'rgw/cloud-transition/{cluster', 'rgw/dbstore/{cluster', 's3tests-branch', 'supported-random-distro$/{centos_latest}', 'tasks/cloud_transition_s3tests}', 'tasks/rgw_s3tests}']
2023-09-18T17:18:04.824 INFO:scrape:
2023-09-18T17:18:04.824 INFO:scrape:Failure: rgw multisite test failures
2023-09-18T17:18:04.824 INFO:scrape:1 jobs: ['7397677']
2023-09-18T17:18:04.825 INFO:scrape:suites: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'supported-random-distro$/{ubuntu_latest}', 'tasks/test_multi}']
2023-09-18T17:18:04.825 INFO:scrape:
2023-09-18T17:18:04.826 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f091ebce8dd]
2023-09-18T17:18:04.826 INFO:scrape:ceph version 18.0.0-5652-g74e83480 (74e834800f96af4850dac39e00acc2e6c3080eb1) reef (dev)
 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f091d784520]
 2: pthread_kill()
 3: raise()
 4: abort()
 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f091ebce8dd]
 6: /usr/lib/ceph/libceph-common.so.2(+0x163a37) [0x7f091ebcea37]
 7: (std::__detail::__variant::__gen_vtable_impl<std::__detail::__variant::_Multi_array<std::__detail::__variant::__deduce_visit_result<int> (*)(FaultInjector<std::basic_string_view<char, std::char_traits<char> > >::check(std::basic_string_view<char, std::char_traits<char> > const&) const::visitor&&, std::variant<std::monostate, InjectAbort, InjectError, InjectDelay> const&)>, std::integer_sequence<unsigned long, 1ul> >::__visit_invoke(FaultInjector<std::basic_string_view<char, std::char_traits<char> > >::check(std::basic_string_view<char, std::char_traits<char> > const&) const::visitor&&, std::variant<std::monostate, InjectAbort, InjectError, InjectDelay> const&)+0x41) [0x558d452a3b41]
 8: radosgw-admin(+0x83fd51) [0x558d452aed51]
 9: (RGWBucketReshard::execute(int, FaultInjector<std::basic_string_view<char, std::char_traits<char> > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x24b) [0x558d452b34cb]
 10: main()
 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f091d76bd90]
 12: __libc_start_main()
 13: _start()
2023-09-18T17:18:04.826 INFO:scrape:1 jobs: ['7397673']
2023-09-18T17:18:04.827 INFO:scrape:suites: ['clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{cls', 'validater/valgrind}', 'versioning}']
2023-09-18T17:18:04.827 INFO:scrape:
2023-09-18T17:18:04.830 INFO:teuthology.results:Sending results to mkogan@redhat.com: Test Run: mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/
failed:  9
dead:    1
running: 0
waiting: 0
queued:  0
passed:  60


Fail
=================================================================
[7397634]  rgw/cloud-transition/{cluster ignore-pg-availability overrides s3tests-branch supported-random-distro$/{centos_latest} tasks/cloud_transition_s3tests}
-----------------------------------------------------------------
time:   00:26:41
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397634/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397634/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1ed132ff63e64c7191c9f054a9fa5752

    Command failed (s3 tests against rgw) on smithi031 with status 1: "source
    /home/ubuntu/cephtest/tox-venv/bin/activate && cd
    /home/ubuntu/cephtest/s3-tests-client.0 &&
    S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not
    fails_on_rgw and not lifecycle_expiration and not test_of_sts and not
    webidentity_test and not fails_with_subdomain and not sse_s3 and
    cloud_transition'"

[7397637]  rgw/dbstore/{cluster ignore-pg-availability overrides s3tests-branch supported-random-distro$/{centos_latest} tasks/rgw_s3tests}
-----------------------------------------------------------------
time:   00:27:55
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397637/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397637/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b48023d1b5984025bbe78d8f212dad7c

    Command failed (s3 tests against rgw) on smithi050 with status 1: "source
    /home/ubuntu/cephtest/tox-venv/bin/activate && cd
    /home/ubuntu/cephtest/s3-tests-client.0 &&
    S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not
    fails_on_rgw and not lifecycle_expiration and not test_of_sts and not
    webidentity_test and not fails_with_subdomain and not sse_s3 and not
    fails_on_dbstore'"

[7397642]  rgw/notifications/{beast bluestore-bitmap fixed-2 ignore-pg-availability overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}}
-----------------------------------------------------------------
time:   00:42:56
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397642/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397642/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bceb960fcf4a417889ecaf90245a9db8

    Command failed (bucket notification tests against different endpoints) on
    smithi125 with status 1: "BNTESTS_CONF=/home/ubuntu/cephtest/ceph/src/test/
    rgw/bucket_notification/bn-tests.client.0.conf /home/ubuntu/cephtest/ceph/s
    rc/test/rgw/bucket_notification/virtualenv/bin/python -m nose -s
    /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py -v
    -a '!kafka_test,!amqp_test,!amqp_ssl_test,!kafka_security_test,!modificatio
    n_required,!manual_test'"

[7397645]  rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast s3tests-branch supported-random-distro$/{centos_latest} tasks/{0-install 1-keycloak 2-s3tests}}
-----------------------------------------------------------------
time:   00:15:23
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397645/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397645/
sentry: https://sentry.ceph.com/organizations/ceph/?query=69c8d98ef4864b3389c346570ca6d0b1

    Command failed on smithi071 with status 127: 'cd
    /home/ubuntu/cephtest/keycloak-11.0.0 && cd bin && ./jboss-cli.sh
    --file=adapter-elytron-install-offline.cli'

[7397655]  rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   01:03:49
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397655/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397655/

    saw valgrind issues

[7397657]  rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest}
-----------------------------------------------------------------
time:   00:14:27
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397657/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397657/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7001b95771d94d3b877050b5398ae5ba

    Command failed on smithi029 with status 100:
    'DEBIAN_FRONTEND=noninteractive sudo -E apt-get -y --force-yes install
    python-dev'

[7397660]  rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   01:04:59
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397660/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397660/

    saw valgrind issues

[7397673]  rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   01:18:16
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397673/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397673/

    saw valgrind issues

[7397677]  rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{ubuntu_latest} tasks/test_multi}
-----------------------------------------------------------------
time:   02:18:23
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397677/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397677/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4dab59ffbd1641888c8cc8251873c0fd

    rgw multisite test failures



Dead
=================================================================
[7397641]  rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zones supported-random-distro$/{centos_latest} tasks/test_multi}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397641/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397641/

    Error reimaging machines: reached maximum tries (101) after waiting for 600
    seconds



Pass
=================================================================
[7397635] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest}
time:   00:34:48
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397635/

[7397636] rgw/d4n/{cluster ignore-pg-availability overrides supported-random-distro$/{ubuntu_latest} tasks/rgw_d4ntests}
time:   00:20:58
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397636/

[7397638] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default ignore-pg-availability overrides s3a-hadoop}
time:   00:23:33
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397638/

[7397639] rgw/lifecycle/{cluster ignore-pg-availability overrides s3tests-branch supported-random-distro$/{centos_latest} tasks/rgw_s3tests}
time:   00:20:07
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397639/

[7397640] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_bucket_quota ubuntu_latest}
time:   00:20:25
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397640/

[7397643] rgw/service-token/{clusters/fixed-1 frontend/beast ignore-pg-availability overrides tasks/service-token ubuntu_latest}
time:   00:18:37
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397643/

[7397644] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{ubuntu_latest}}
time:   00:21:41
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397644/

[7397646] rgw/tempest/{clusters/fixed-1 frontend/beast ignore-pg-availability overrides tasks/rgw_tempest ubuntu_latest}
time:   00:26:44
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397646/

[7397647] rgw/thrash/{clusters/fixed-2 frontend/beast ignore-pg-availability install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota}
time:   00:20:39
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397647/

[7397648] rgw/tools/{centos_latest cluster ignore-pg-availability tasks}
time:   02:41:45
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397648/

[7397649] rgw/upgrade/{1-install/pacific/{distro$/{centos_8.stream} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides}
time:   00:24:35
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397649/

[7397650] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
time:   00:43:12
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397650/

[7397651] rgw/website/{clusters/fixed-2 frontend/beast http ignore-pg-availability overrides s3tests-branch tasks/s3tests-website ubuntu_latest}
time:   00:27:09
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397651/

[7397652] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
time:   01:43:41
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397652/

[7397653] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_multipart_upload ubuntu_latest}
time:   00:18:39
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397653/

[7397654] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
time:   00:42:31
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397654/

[7397656] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_ragweed ubuntu_latest}
time:   00:20:49
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397656/

[7397658] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
time:   00:44:44
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397658/

[7397659] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_s3tests ubuntu_latest}
time:   00:24:54
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397659/

[7397661] rgw/thrash/{clusters/fixed-2 frontend/beast ignore-pg-availability install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload}
time:   00:17:59
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397661/

[7397662] rgw/upgrade/{1-install/quincy/{distro$/{ubuntu_20.04} install overrides} 2-setup 3-upgrade-sequence/rgws-then-osds cluster frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides}
time:   00:19:06
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397662/

[7397663] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
time:   00:47:19
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397663/

[7397664] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_user_quota ubuntu_latest}
time:   00:18:48
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397664/

[7397665] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
time:   01:17:59
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397665/

[7397666] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest}
time:   00:27:20
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397666/

[7397667] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_bucket_quota ubuntu_latest}
time:   00:20:02
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397667/

[7397668] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_latest}}
time:   00:17:11
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397668/

[7397669] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
time:   00:44:35
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397669/

[7397670] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
time:   01:12:12
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397670/

[7397671] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_multipart_upload ubuntu_latest}
time:   00:15:53
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397671/

[7397672] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
time:   00:43:48
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397672/

[7397674] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_ragweed ubuntu_latest}
time:   00:17:33
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397674/

[7397675] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest}
time:   00:25:39
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397675/

[7397676] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 ignore-pg-availability overrides s3a-hadoop}
time:   00:20:36
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397676/

[7397678] rgw/thrash/{clusters/fixed-2 frontend/beast ignore-pg-availability install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests}
time:   00:23:07
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397678/

[7397679] rgw/upgrade/{1-install/quincy/{distro$/{centos_latest} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides}
time:   00:17:55
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397679/

[7397680] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
time:   00:42:50
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397680/

[7397681] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_s3tests ubuntu_latest}
time:   00:23:35
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397681/

[7397682] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
time:   01:47:57
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397682/

[7397683] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
time:   00:35:07
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397683/

[7397684] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_user_quota ubuntu_latest}
time:   00:16:03
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397684/

[7397685] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
time:   01:01:17
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397685/

[7397686] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest}
time:   00:25:32
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397686/

[7397687] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_bucket_quota ubuntu_latest}
time:   00:16:02
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397687/

[7397688] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{centos_latest}}
time:   00:17:44
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397688/

[7397689] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
time:   00:33:57
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397689/

[7397690] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
time:   01:02:19
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397690/

[7397691] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_multipart_upload ubuntu_latest}
time:   00:15:27
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397691/

[7397692] rgw/thrash/{clusters/fixed-2 frontend/beast ignore-pg-availability install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota}
time:   00:15:48
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397692/

[7397693] rgw/upgrade/{1-install/pacific/{distro$/{ubuntu_20.04} install overrides} 2-setup 3-upgrade-sequence/rgws-then-osds cluster frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides}
time:   00:17:32
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397693/

[7397694] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
time:   00:36:51
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397694/

[7397695] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
time:   00:52:01
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397695/

[7397696] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_ragweed ubuntu_latest}
time:   00:18:25
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397696/

[7397697] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest}
time:   00:25:37
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397697/

[7397698] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
time:   00:36:13
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397698/

[7397699] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_s3tests ubuntu_latest}
time:   00:23:01
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397699/

[7397700] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
time:   00:48:42
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397700/

[7397701] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
time:   00:36:20
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397701/

[7397702] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_user_quota ubuntu_latest}
time:   00:14:47
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397702/

[7397703] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
time:   01:18:40
info:   http://pulpito.ceph.com/mkogan-2023-09-18_14:17:13-rgw-rgw-wip-t62141-valg-supp_i005-distro-default-smithi/7397703/