2021-10-01T15:04:19.660 INFO:root:teuthology version: 1.1.0-f08b273e
2021-10-01T15:04:19.662 INFO:teuthology.results:subset: '1/2'
2021-10-01T15:04:19.662 INFO:teuthology.results:seed: '5983'
2021-10-01T15:29:28.360 INFO:root:teuthology version: 1.1.0-f08b273e
2021-10-01T15:29:28.388 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-10-02T03:22:20.243 INFO:teuthology.results:Tests finished! gathering results...
2021-10-02T03:22:26.176 INFO:scrape:Found 68 jobs
2021-10-02T03:22:38.238 INFO:scrape:Found 10 distinct failure reasons
2021-10-02T03:22:38.239 INFO:scrape:Failure: Command failed on smithi184 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.0.0-7958-gefc11bb1-1focal cephadm=17.0.0-7958-gefc11bb1-1focal ceph-mds=17.0.0-7958-gefc11bb1-1focal ceph-mgr=17.0.0-7958-gefc11bb1-1focal ceph-common=17.0.0-7958-gefc11bb1-1focal ceph-fuse=17.0.0-7958-gefc11bb1-1focal ceph-test=17.0.0-7958-gefc11bb1-1focal ceph-volume=17.0.0-7958-gefc11bb1-1focal radosgw=17.0.0-7958-gefc11bb1-1focal python3-rados=17.0.0-7958-gefc11bb1-1focal python3-rgw=17.0.0-7958-gefc11bb1-1focal python3-cephfs=17.0.0-7958-gefc11bb1-1focal python3-rbd=17.0.0-7958-gefc11bb1-1focal libcephfs2=17.0.0-7958-gefc11bb1-1focal libcephfs-dev=17.0.0-7958-gefc11bb1-1focal librados2=17.0.0-7958-gefc11bb1-1focal librbd1=17.0.0-7958-gefc11bb1-1focal rbd-fuse=17.0.0-7958-gefc11bb1-1focal'
2021-10-02T03:22:38.240 INFO:scrape:33 jobs
2021-10-02T03:22:38.241 INFO:scrape:suites intersection: []
2021-10-02T03:22:38.241 INFO:scrape:suites union: ['1-ceph-install/install', '2-kms/barbican', '2-kms/kmip', '2-kms/testing', '2-kms/vault_transit', '3-rgw/rgw', '4-tests/{s3tests}', 'frontend/beast', 'hadoop/default', 'hadoop/v32', 'http', 'ignore-pg-availability', 'install', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'rgw/crypt/{0-cluster/fixed-1', 'rgw/hadoop-s3a/{clusters/fixed-2', 'rgw/multifs/{clusters/fixed-2', 'rgw/singleton/{all/radosgw-admin', 'rgw/tempest/{clusters/fixed-1', 'rgw/thrash/{clusters/fixed-2', 'rgw/website/{clusters/fixed-2', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3a-hadoop}', 'supported-random-distro$/{ubuntu_latest}}', 'tasks/rgw_bucket_quota', 'tasks/rgw_multipart_upload', 'tasks/rgw_ragweed', 'tasks/rgw_s3tests', 'tasks/rgw_tempest', 'tasks/rgw_user_quota', 'tasks/s3tests-website', 'thrasher/default', 'thrashosds-health', 'ubuntu_latest', 'ubuntu_latest}', 'workload/rgw_bucket_quota}', 'workload/rgw_multipart_upload}', 'workload/rgw_s3tests}', 'workload/rgw_user_quota}']
2021-10-02T03:22:38.241 INFO:scrape:
2021-10-02T03:22:38.242 INFO:scrape:Crash: Command failed (workunit test cls/test_cls_2pc_queue.sh) on smithi059 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=efc11bb151d89ad06b102b29677829466295d026 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh'
2021-10-02T03:22:38.242 INFO:scrape:ceph version 17.0.0-7958-gefc11bb1 (efc11bb151d89ad06b102b29677829466295d026) quincy (dev)
 1: /lib64/libpthread.so.0(+0x12b20) [0x7ff00a814b20]
 2: gsignal()
 3: abort()
 4: /lib64/libc.so.6(+0x21c89) [0x7ff009460c89]
 5: /lib64/libc.so.6(+0x2fa76) [0x7ff00946ea76]
 6: ceph-osd(+0x7c7695) [0x55ae9c187695]
 7: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0xe46) [0x55ae9c1a8666]
 8: (PrimaryLogPG::simple_opc_submit(std::unique_ptr<PrimaryLogPG::OpContext, std::default_delete<PrimaryLogPG::OpContext> >)+0x120) [0x55ae9c1aa750]
 9: (PrimaryLogPG::AwaitAsyncWork::react(PrimaryLogPG::DoSnapWork const&)+0x766) [0x55ae9c1ec666]
 10: (boost::statechart::simple_state<PrimaryLogPG::AwaitAsyncWork, PrimaryLogPG::Trimming, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::react_impl(boost::statechart::event_base const&, void const*)+0x121) [0x55ae9c270a31]
 11: (boost::statechart::state_machine<PrimaryLogPG::SnapTrimmer, PrimaryLogPG::NotTrimming, std::allocator<boost::statechart::none>, boost::statechart::null_exception_translator>::process_event(boost::statechart::event_base const&)+0x74) [0x55ae9c239654]
 12: (PrimaryLogPG::snap_trimmer(unsigned int)+0x17a) [0x55ae9c19e5da]
 13: (ceph::osd::scheduler::PGSnapTrim::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x1f) [0x55ae9c35a9af]
 14: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xac8) [0x55ae9c0b6e48]
 15: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x5c4) [0x55ae9c7a5c64]
 16: (ShardedThreadPool::WorkThreadSharded::entry()+0x14) [0x55ae9c7a6754]
 17: /lib64/libpthread.so.0(+0x814a) [0x7ff00a80a14a]
 18: clone()
2021-10-02T03:22:38.243 INFO:scrape:19 jobs: ['6416382', '6416401', '6416361', '6416404', '6416381', '6416349', '6416369', '6416347', '6416385', '6416398', '6416389', '6416379', '6416351', '6416402', '6416371', '6416390', '6416359', '6416367', '6416363']
2021-10-02T03:22:38.243 INFO:scrape:suites intersection: ['centos_latest', 'clusters/fixed-2', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'overrides', 'ragweed', 'reshard', 'rgw/verify/{0-install', 's3tests-java', 's3tests}', 'tasks/{cls']
2021-10-02T03:22:38.244 INFO:scrape:suites union: ['centos_latest', 'clusters/fixed-2', 'datacache/no_datacache', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{cls', 'validater/lockdep}', 'validater/valgrind}']
2021-10-02T03:22:38.244 INFO:scrape:
2021-10-02T03:22:38.245 INFO:scrape:Failure: 
2021-10-02T03:22:38.245 INFO:scrape:4 jobs: ['6416388', '6416377', '6416356', '6416397']
2021-10-02T03:22:38.246 INFO:scrape:suites intersection: ['frontend/beast', 'overrides', 'rgw/singleton/{all/radosgw-admin']
2021-10-02T03:22:38.246 INFO:scrape:suites union: ['frontend/beast', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'rgw/singleton/{all/radosgw-admin', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 'supported-random-distro$/{centos_8.stream}}', 'supported-random-distro$/{rhel_8}}']
2021-10-02T03:22:38.247 INFO:scrape:
2021-10-02T03:22:38.247 INFO:scrape:Failure: rgw multisite test failures
2021-10-02T03:22:38.248 INFO:scrape:2 jobs: ['6416376', '6416340']
2021-10-02T03:22:38.248 INFO:scrape:suites intersection: ['frontend/beast', 'ignore-pg-availability', 'omap_limits', 'overrides', 'rgw/multisite/{clusters', 'supported-random-distro$/{centos_8.stream}', 'tasks/test_multi}']
2021-10-02T03:22:38.248 INFO:scrape:suites union: ['frontend/beast', 'ignore-pg-availability', 'omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'supported-random-distro$/{centos_8.stream}', 'tasks/test_multi}']
2021-10-02T03:22:38.249 INFO:scrape:
2021-10-02T03:22:38.249 INFO:scrape:Failure: Command failed (s3 tests against rgw) on smithi180 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!test_of_sts,!webidentity_test,!fails_with_subdomain'"
2021-10-02T03:22:38.250 INFO:scrape:2 jobs: ['6416386', '6416374']
2021-10-02T03:22:38.250 INFO:scrape:suites intersection: ['1-ceph-install/install', '3-rgw/rgw', '4-tests/{s3tests}', 'ignore-pg-availability', 'rgw/crypt/{0-cluster/fixed-1']
2021-10-02T03:22:38.251 INFO:scrape:suites union: ['1-ceph-install/install', '2-kms/vault_kv', '2-kms/vault_old', '3-rgw/rgw', '4-tests/{s3tests}', 'ignore-pg-availability', 'rgw/crypt/{0-cluster/fixed-1', 'supported-random-distro$/{centos_8.stream}}', 'supported-random-distro$/{rhel_8}}']
2021-10-02T03:22:38.251 INFO:scrape:
2021-10-02T03:22:38.251 INFO:scrape:Crash: Command failed (workunit test rgw/run-reshard.sh) on smithi120 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=efc11bb151d89ad06b102b29677829466295d026 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh'
2021-10-02T03:22:38.252 INFO:scrape:ceph version 17.0.0-7958-gefc11bb1 (efc11bb151d89ad06b102b29677829466295d026) quincy (dev)
 1: /lib64/libpthread.so.0(+0x12dc0) [0x7f1f8bccddc0]
 2: gsignal()
 3: abort()
 4: /lib64/libc.so.6(+0x21bc9) [0x7f1f8a923bc9]
 5: /lib64/libc.so.6(+0x2fe96) [0x7f1f8a931e96]
 6: ceph-osd(+0x7c7695) [0x564bbf625695]
 7: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0xe46) [0x564bbf646666]
 8: (PrimaryLogPG::simple_opc_submit(std::unique_ptr<PrimaryLogPG::OpContext, std::default_delete<PrimaryLogPG::OpContext> >)+0x120) [0x564bbf648750]
 9: (PrimaryLogPG::AwaitAsyncWork::react(PrimaryLogPG::DoSnapWork const&)+0x766) [0x564bbf68a666]
 10: (boost::statechart::simple_state<PrimaryLogPG::AwaitAsyncWork, PrimaryLogPG::Trimming, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::react_impl(boost::statechart::event_base const&, void const*)+0x121) [0x564bbf70ea31]
 11: (boost::statechart::state_machine<PrimaryLogPG::SnapTrimmer, PrimaryLogPG::NotTrimming, std::allocator<boost::statechart::none>, boost::statechart::null_exception_translator>::process_event(boost::statechart::event_base const&)+0x74) [0x564bbf6d7654]
 12: (PrimaryLogPG::snap_trimmer(unsigned int)+0x17a) [0x564bbf63c5da]
 13: (ceph::osd::scheduler::PGSnapTrim::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x1f) [0x564bbf7f89af]
 14: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xac8) [0x564bbf554e48]
 15: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x5c4) [0x564bbfc43c64]
 16: (ShardedThreadPool::WorkThreadSharded::entry()+0x14) [0x564bbfc44754]
 17: /lib64/libpthread.so.0(+0x82de) [0x7f1f8bcc32de]
 18: clone()
2021-10-02T03:22:38.252 INFO:scrape:1 jobs: ['6416394']
2021-10-02T03:22:38.253 INFO:scrape:suites: ['centos_latest', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 's3tests-java', 's3tests}', 'sharding$/{default}', 'striping$/{stripe-equals-chunk}', 'tasks/{cls', 'validater/valgrind}']
2021-10-02T03:22:38.253 INFO:scrape:
2021-10-02T03:22:38.254 INFO:scrape:Dead: b'2021-10-02T03:17:15.576 DEBUG:teuthology.exit:Finished running handlers'
2021-10-02T03:22:38.254 INFO:scrape:ceph version 17.0.0-7958-gefc11bb1 (efc11bb151d89ad06b102b29677829466295d026) quincy (dev)
 1: /lib64/libpthread.so.0(+0x12dc0) [0x7f5a34fcadc0]
 2: gsignal()
 3: abort()
 4: /lib64/libc.so.6(+0x21bc9) [0x7f5a33c20bc9]
 5: /lib64/libc.so.6(+0x2fe96) [0x7f5a33c2ee96]
 6: ceph-osd(+0x7c7695) [0x55941533d695]
 7: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0xe46) [0x55941535e666]
 8: (PrimaryLogPG::simple_opc_submit(std::unique_ptr<PrimaryLogPG::OpContext, std::default_delete<PrimaryLogPG::OpContext> >)+0x120) [0x559415360750]
 9: (PrimaryLogPG::AwaitAsyncWork::react(PrimaryLogPG::DoSnapWork const&)+0x766) [0x5594153a2666]
 10: (boost::statechart::simple_state<PrimaryLogPG::AwaitAsyncWork, PrimaryLogPG::Trimming, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::react_impl(boost::statechart::event_base const&, void const*)+0x121) [0x559415426a31]
 11: (boost::statechart::state_machine<PrimaryLogPG::SnapTrimmer, PrimaryLogPG::NotTrimming, std::allocator<boost::statechart::none>, boost::statechart::null_exception_translator>::process_event(boost::statechart::event_base const&)+0x74) [0x5594153ef654]
 12: (PrimaryLogPG::snap_trimmer(unsigned int)+0x17a) [0x5594153545da]
 13: (ceph::osd::scheduler::PGSnapTrim::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x1f) [0x5594155109af]
 14: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xac8) [0x55941526ce48]
 15: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x5c4) [0x55941595bc64]
 16: (ShardedThreadPool::WorkThreadSharded::entry()+0x14) [0x55941595c754]
 17: /lib64/libpthread.so.0(+0x82de) [0x7f5a34fc02de]
 18: clone()
2021-10-02T03:22:38.255 INFO:scrape:2 jobs: ['6416353', '6416357']
2021-10-02T03:22:38.255 INFO:scrape:suites intersection: ['centos_latest', 'clusters/fixed-2', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'overrides', 'ragweed', 'reshard', 'rgw/verify/{0-install', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'tasks/{cls']
2021-10-02T03:22:38.256 INFO:scrape:suites union: ['centos_latest', 'clusters/fixed-2', 'datacache/no_datacache', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'tasks/{cls', 'validater/lockdep}', 'validater/valgrind}']
2021-10-02T03:22:38.256 INFO:scrape:
2021-10-02T03:22:38.256 INFO:scrape:Crash: "2021-10-01T17:57:15.626137+0000 mon.a (mon.0) 473 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log
2021-10-02T03:22:38.257 INFO:scrape:ceph version 17.0.0-7958-gefc11bb1 (efc11bb151d89ad06b102b29677829466295d026) quincy (dev)
 1: /lib64/libpthread.so.0(+0x12b20) [0x7f4ccbf78b20]
 2: gsignal()
 3: abort()
 4: /lib64/libc.so.6(+0x21c89) [0x7f4ccabc4c89]
 5: /lib64/libc.so.6(+0x2fa76) [0x7f4ccabd2a76]
 6: ceph-osd(+0x7c7695) [0x55fbc5a9e695]
 7: (PrimaryLogPG::finish_ctx(PrimaryLogPG::OpContext*, int, int)+0x14f5) [0x55fbc5ac9245]
 8: (PrimaryLogScrub::scrub_snapshot_metadata(ScrubMap&, std::map<hobject_t, std::pair<std::optional<unsigned int>, std::optional<unsigned int> >, std::less<hobject_t>, std::allocator<std::pair<hobject_t const, std::pair<std::optional<unsigned int>, std::optional<unsigned int> > > > > const&)+0x3ada) [0x55fbc5bdcdca]
 9: (PgScrubber::scrub_compare_maps()+0x1b97) [0x55fbc5bbce67]
 10: (PgScrubber::maps_compare_n_cleanup()+0xd) [0x55fbc5bbd05d]
 11: (Scrub::WaitReplicas::react(Scrub::GotReplicas const&)+0x2ec) [0x55fbc5bfa2dc]
 12: (boost::statechart::simple_state<Scrub::WaitReplicas, Scrub::ActiveScrubbing, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::react_impl(boost::statechart::event_base const&, void const*)+0xe1) [0x55fbc5c016d1]
 13: (boost::statechart::state_machine<Scrub::ScrubMachine, Scrub::NotActive, std::allocator<boost::statechart::none>, boost::statechart::null_exception_translator>::process_event(boost::statechart::event_base const&)+0x74) [0x55fbc5bcda04]
 14: (PgScrubber::send_replica_maps_ready(unsigned int)+0x2fd) [0x55fbc5bc463d]
 15: (PG::forward_scrub_event(void (ScrubPgIF::*)(unsigned int), unsigned int, std::basic_string_view<char, std::char_traits<char> >)+0x82) [0x55fbc5a64062]
 16: (ceph::osd::scheduler::PGScrubGotReplMaps::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x2f) [0x55fbc5c71ecf]
 17: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xac8) [0x55fbc59cde48]
 18: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x5c4) [0x55fbc60bcc64]
 19: (ShardedThreadPool::WorkThreadSharded::entry()+0x14) [0x55fbc60bd754]
 20: /lib64/libpthread.so.0(+0x814a) [0x7f4ccbf6e14a]
 21: clone()
2021-10-02T03:22:38.257 INFO:scrape:1 jobs: ['6416346']
2021-10-02T03:22:38.258 INFO:scrape:suites: ['cluster', 'rgw/tools/{centos_latest', 'tasks}']
2021-10-02T03:22:38.258 INFO:scrape:
2021-10-02T03:22:38.259 INFO:scrape:Failure: "2021-10-01T15:40:51.859275+0000 mon.a (mon.0) 117 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log
2021-10-02T03:22:38.259 INFO:scrape:1 jobs: ['6416372']
2021-10-02T03:22:38.259 INFO:scrape:suites: ['centos_latest', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{cls', 'validater/valgrind}']
2021-10-02T03:22:38.260 INFO:scrape:
2021-10-02T03:22:38.260 INFO:scrape:Failure: reached maximum tries (90) after waiting for 540 seconds
2021-10-02T03:22:38.261 INFO:scrape:1 jobs: ['6416393']
2021-10-02T03:22:38.261 INFO:scrape:suites: ['centos_latest', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/filestore-xfs', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec-profile', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{cls', 'validater/lockdep}']
2021-10-02T03:22:38.262 INFO:scrape: