2024-01-13T08:58:18.957 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-13T08:58:18.957 INFO:teuthology.results:subset: '0/999' 2024-01-13T08:58:18.957 INFO:teuthology.results:seed: '4203' 2024-01-13T09:01:37.001 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-13T09:01:37.004 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-01-13T10:42:44.854 INFO:teuthology.results:Tests finished! gathering results... 2024-01-13T10:42:45.271 INFO:scrape:Found 5 jobs 2024-01-13T10:42:45.438 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-607-g2fca3ee6 (2fca3ee615243c049c12dddfad2b5e58a398cdcc) squid (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f9f2e78030d] , 2: ceph_test_rados(+0x284f9) [0x5645344cb4f9] 2024-01-13T10:42:45.438 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2024-01-13T10:42:45.467 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-607-g2fca3ee6 (2fca3ee615243c049c12dddfad2b5e58a398cdcc) squid (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f9f2e78030d] , 2: ceph_test_rados(+0x284f9) [0x5645344cb4f9] 2024-01-13T10:42:45.468 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2024-01-13T10:42:45.508 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-607-g2fca3ee6 (2fca3ee615243c049c12dddfad2b5e58a398cdcc) squid (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f9f2e78030d] , 2: ceph_test_rados(+0x284f9) [0x5645344cb4f9] 2024-01-13T10:42:45.509 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2024-01-13T10:42:46.526 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-607-g2fca3ee6 (2fca3ee615243c049c12dddfad2b5e58a398cdcc) squid (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f5641b8030d] , 2: ceph_test_rados(+0x284f9) [0x5609f9ec84f9] 2024-01-13T10:42:46.526 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2024-01-13T10:42:47.457 WARNING:scrape:Ignoring backtrace that exceeds MAX_BACKTRACE_LINES: ceph version 19.0.0-607-g2fca3ee6 (2fca3ee615243c049c12dddfad2b5e58a398cdcc) squid (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f0b0f1d7aa5] , 2: ceph_test_rados(+0x27b6f) [0x564fcc33ab6f] 2024-01-13T10:42:47.458 WARNING:scrape:Either the backtrace is too long or we did a bad job of checking for end of backtrace! 2024-01-13T10:42:47.497 INFO:scrape:Found 2 distinct failure reasons 2024-01-13T10:42:47.497 INFO:scrape:Failure: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --low_tier_pool low_tier --max-ops 3000 --objects 150 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 360 --op read 100 --op write 10 --op delete 10 --op write_excl 10 --pool base_tier' 2024-01-13T10:42:47.497 INFO:scrape:3 jobs: ['7516070', '7516069', '7516073'] 2024-01-13T10:42:47.497 INFO:scrape:suites intersection: ['1-pg-log-overrides/normal_pg_log', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'mon_election/classic', 'openstack}', 'rados', 'rados:thrash/{0-size-min-size-overrides/2-size-2-min-size', 'thrashosds-health', 'workloads/dedup-daemon}'] 2024-01-13T10:42:47.497 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-active-recovery}', '2-recovery-overrides/{more-async-recovery}', '2-recovery-overrides/{more-partial-recovery}', '3-scrub-overrides/{default}', '3-scrub-overrides/{max-simultaneous-scrubs-2}', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'mon_election/classic', 'msgr-failures/fastclose', 'msgr-failures/osd-delay', 'msgr/async', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados:thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/mapgap', 'thrashers/none', 'thrashosds-health', 'workloads/dedup-daemon}'] 2024-01-13T10:42:47.498 INFO:scrape: 2024-01-13T10:42:47.498 INFO:scrape:Failure: timed out getting correct exitstatus 2024-01-13T10:42:47.498 INFO:scrape:2 jobs: ['7516071', '7516072'] 2024-01-13T10:42:47.498 INFO:scrape:suites intersection: ['1-pg-log-overrides/normal_pg_log', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'mon_election/classic', 'openstack}', 'rados', 'rados:thrash/{0-size-min-size-overrides/2-size-2-min-size', 'thrashosds-health', 'workloads/dedup-daemon}'] 2024-01-13T10:42:47.498 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', '2-recovery-overrides/{more-active-recovery}', '3-scrub-overrides/{default}', '3-scrub-overrides/{max-simultaneous-scrubs-2}', 'backoff/normal', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'mon_election/classic', 'msgr-failures/fastclose', 'msgr-failures/osd-delay', 'msgr/async', 'msgr/async-v1only', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados:thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/careful', 'thrashers/mapgap', 'thrashosds-health', 'workloads/dedup-daemon}'] 2024-01-13T10:42:47.498 INFO:scrape: