2026-02-21T12:02:24.251 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-21T12:02:24.252 INFO:teuthology.results:subset: '1/10000'
2026-02-21T12:02:24.252 INFO:teuthology.results:seed: '2477'
2026-02-21T12:14:41.123 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-21T12:14:41.129 INFO:teuthology.results:Waiting up to 7200 seconds for tests to finish...
2026-02-21T13:23:46.522 INFO:teuthology.results:Tests finished! gathering results...
2026-02-21T13:23:47.855 INFO:scrape:Found 22 jobs
2026-02-21T13:23:47.857 INFO:scrape:Found 3 distinct failure reasons
2026-02-21T13:23:47.857 INFO:scrape:Failure: reached maximum tries (500) after waiting for 3000 seconds
2026-02-21T13:23:47.857 INFO:scrape:1 jobs: ['62664']
2026-02-21T13:23:47.857 INFO:scrape:suites: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-active-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-3}', 'backoff/peering_and_degraded', 'base', 'ceph', 'clusters/{fixed-4}', 'crc-failures/default', 'd-balancer/on', 'mem$/{normal-2}', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'objectstore/{bluestore/{alloc$/{hybrid}', 'onode-segment$/{512K}', 'rados', 'rados:thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_latest}', 'thrashers/careful_host', 'thrashosds-health', 'v1}}}}', 'workloads/radosbench-high-concurrency}', 'write$/{v1/{compr$/{yes$/{lz4}}']
2026-02-21T13:23:47.857 INFO:scrape:
2026-02-21T13:23:47.857 INFO:scrape:Failure: The WatchDog barked due to Daemon ceph.osd.12 has failed
2026-02-21T13:23:47.857 INFO:scrape:1 jobs: ['62682']
2026-02-21T13:23:47.857 INFO:scrape:suites: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-async-partial-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-1}', 'backoff/normal', 'base', 'ceph', 'clusters/{fixed-4}', 'crc-failures/default', 'd-balancer/upmap-read', 'mem$/{normal-1}', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'msgr/async-v1only', 'objectstore/{bluestore/{alloc$/{btree}', 'onode-segment$/{none}', 'rados', 'rados:thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_latest}', 'thrashers/mapgap_host', 'thrashosds-health', 'v1}}}}', 'workloads/pool-snaps-few-objects}', 'write$/{v1/{compr$/{no$/{no}}']
2026-02-21T13:23:47.857 INFO:scrape:
2026-02-21T13:23:47.857 INFO:scrape:Failure: Command failed (workunit test rados/test.sh) on trial187 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7eb1e7b478bae35f04b0b595f6e5c1e0a5b1f998 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'
2026-02-21T13:23:47.857 INFO:scrape:1 jobs: ['62683']
2026-02-21T13:23:47.857 INFO:scrape:suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-async-partial-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-5}', 'backoff/peering', 'base', 'ceph', 'clusters/{fixed-4}', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'mem$/{low}', 'mon_election/classic', 'msgr-failures/fastclose', 'msgr/async-v2only', 'objectstore/{bluestore/{alloc$/{stupid}', 'onode-segment$/{512K-onoff}', 'rados', 'rados:thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{centos_latest}', 'thrashers/morepggrow', 'thrashosds-health', 'v2}}}}', 'workloads/rados_api_tests}', 'write$/{v2/{compr$/{yes$/{zstd}}']
2026-02-21T13:23:47.857 INFO:scrape:
2026-02-21T13:23:47.857 INFO:teuthology.results:Sending results to xxhdx1985126@gmail.com: Test Run: xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial
=================================================================
info:    https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/
logs:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/
failed:  3
dead:    0
running: 0
waiting: 0
queued:  0
passed:  19


Fail
=================================================================
[62664]  rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-4} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/{bluestore/{alloc$/{hybrid} base mem$/{normal-2} onode-segment$/{512K} write$/{v1/{compr$/{yes$/{lz4}} v1}}}} rados supported-random-distro$/{centos_latest} thrashers/careful_host thrashosds-health workloads/radosbench-high-concurrency}
-----------------------------------------------------------------
time:   01:16:11
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62664/
log:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62664/

    reached maximum tries (500) after waiting for 3000 seconds

[62682]  rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4} crc-failures/default d-balancer/upmap-read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/{bluestore/{alloc$/{btree} base mem$/{normal-1} onode-segment$/{none} write$/{v1/{compr$/{no$/{no}} v1}}}} rados supported-random-distro$/{centos_latest} thrashers/mapgap_host thrashosds-health workloads/pool-snaps-few-objects}
-----------------------------------------------------------------
time:   00:21:41
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62682/
log:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62682/

    The WatchDog barked due to Daemon ceph.osd.12 has failed

[62683]  rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-4} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/{bluestore/{alloc$/{stupid} base mem$/{low} onode-segment$/{512K-onoff} write$/{v2/{compr$/{yes$/{zstd}} v2}}}} rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/rados_api_tests}
-----------------------------------------------------------------
time:   00:12:19
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62683/
log:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62683/

    Command failed (workunit test rados/test.sh) on trial187 with status 1:
    'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=7eb1e7b478bae35f04b0b595f6e5c1e0a5b1f998
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'



Pass
=================================================================
[62665] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-4} crc-failures/bad_map_crc_failure d-balancer/read mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/{bluestore/{alloc$/{bitmap} base mem$/{normal-2} onode-segment$/{256K} write$/{v2/{compr$/{yes$/{lz4}} v2}}}} rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/radosbench}
time:   00:27:16
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62665/

[62666] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-4} crc-failures/default d-balancer/upmap-read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/{bluestore/{alloc$/{avl} base mem$/{normal-1} onode-segment$/{512K-onoff} write$/{random/{compr$/{yes$/{zstd}} random}}}} rados supported-random-distro$/{centos_latest} thrashers/default_host thrashosds-health workloads/redirect}
time:   00:19:13
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62666/

[62667] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-4} crc-failures/bad_map_crc_failure d-balancer/read mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/{bluestore/{alloc$/{avl} base mem$/{normal-2} onode-segment$/{512K-onoff} write$/{v2/{compr$/{yes$/{zstd}} v2}}}} rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/set-chunks-read}
time:   00:10:22
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62667/

[62668] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-4} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/{bluestore/{alloc$/{btree} base mem$/{low} onode-segment$/{512K-onoff} write$/{v2/{compr$/{no$/{no}} v2}}}} rados supported-random-distro$/{centos_latest} thrashers/pggrow_host thrashosds-health workloads/small-objects}
time:   00:17:36
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62668/

[62669] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-4} crc-failures/bad_map_crc_failure d-balancer/read mon_election/classic msgr-failures/osd-delay msgr/async objectstore/{bluestore/{alloc$/{hybrid} base mem$/{low} onode-segment$/{512K-onoff} write$/{v2/{compr$/{yes$/{zstd}} v2}}}} rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced}
time:   00:21:44
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62669/

[62670] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-4} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/{bluestore/{alloc$/{avl} base mem$/{normal-2} onode-segment$/{512K-onoff} write$/{v1/{compr$/{no$/{no}} v1}}}} rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/snaps-few-objects}
time:   00:21:50
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62670/

[62671] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-4} crc-failures/bad_map_crc_failure d-balancer/read mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/{bluestore/{alloc$/{stupid} base mem$/{low} onode-segment$/{512K} write$/{v1/{compr$/{yes$/{lz4}} v1}}}} rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/admin_socket_objecter_requests}
time:   00:19:35
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62671/

[62672] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-4} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/{bluestore/{alloc$/{hybrid} base mem$/{low} onode-segment$/{512K} write$/{v1/{compr$/{no$/{no}} v1}}}} rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/dedup-io-snaps}
time:   00:09:17
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62672/

[62673] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/{bluestore/{alloc$/{bitmap} base mem$/{normal-1} onode-segment$/{none} write$/{v1/{compr$/{no$/{no}} v1}}}} rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/pool-snaps-few-objects}
time:   00:20:18
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62673/

[62674] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-4} crc-failures/bad_map_crc_failure d-balancer/read mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/{bluestore/{alloc$/{hybrid} base mem$/{normal-1} onode-segment$/{1M} write$/{random/{compr$/{no$/{no}} random}}}} rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/rados_api_tests}
time:   00:12:22
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62674/

[62675] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v1only objectstore/{bluestore/{alloc$/{stupid} base mem$/{low} onode-segment$/{512K} write$/{v1/{compr$/{no$/{no}} v1}}}} rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/radosbench}
time:   00:47:05
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62675/

[62676] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-4} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/{bluestore/{alloc$/{stupid} base mem$/{normal-1} onode-segment$/{512K-onoff} write$/{random/{compr$/{no$/{no}} random}}}} rados supported-random-distro$/{centos_latest} thrashers/careful_host thrashosds-health workloads/redirect}
time:   00:10:16
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62676/

[62677] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-4} crc-failures/bad_map_crc_failure d-balancer/read mon_election/classic msgr-failures/osd-delay msgr/async objectstore/{bluestore/{alloc$/{stupid} base mem$/{low} onode-segment$/{512K-onoff} write$/{v2/{compr$/{no$/{no}} v2}}}} rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/redirect_promote_tests}
time:   00:08:34
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62677/

[62678] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-4} crc-failures/bad_map_crc_failure d-balancer/read mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/{bluestore/{alloc$/{avl} base mem$/{low} onode-segment$/{none} write$/{random/{compr$/{yes$/{zlib}} random}}}} rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/small-objects-localized}
time:   00:17:07
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62678/

[62679] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/normal ceph clusters/{fixed-4} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/{bluestore/{alloc$/{hybrid} base mem$/{normal-1} onode-segment$/{1M} write$/{v2/{compr$/{no$/{no}} v2}}}} rados supported-random-distro$/{centos_latest} thrashers/pggrow_host thrashosds-health workloads/snaps-few-objects-localized}
time:   00:21:20
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62679/

[62680] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-4} crc-failures/bad_map_crc_failure d-balancer/read mon_election/classic msgr-failures/osd-delay msgr/async-v2only objectstore/{bluestore/{alloc$/{stupid} base mem$/{normal-2} onode-segment$/{512K} write$/{v2/{compr$/{yes$/{zstd}} v2}}}} rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects}
time:   00:21:05
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62680/

[62681] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering ceph clusters/{fixed-4} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/{bluestore/{alloc$/{avl} base mem$/{normal-2} onode-segment$/{1M} write$/{v2/{compr$/{no$/{no}} v2}}}} rados supported-random-distro$/{centos_latest} thrashers/default_host thrashosds-health workloads/dedup-io-mixed}
time:   00:08:32
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62681/

[62684] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/peering_and_degraded ceph clusters/{fixed-4} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async objectstore/{bluestore/{alloc$/{avl} base mem$/{normal-2} onode-segment$/{1M} write$/{random/{compr$/{yes$/{zstd}} random}}}} rados supported-random-distro$/{centos_latest} thrashers/none thrashosds-health workloads/radosbench-high-concurrency}
time:   00:32:25
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62684/

[62685] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering ceph clusters/{fixed-4} crc-failures/default d-balancer/upmap-read mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/{bluestore/{alloc$/{btree} base mem$/{normal-1} onode-segment$/{512K-onoff} write$/{v2/{compr$/{no$/{no}} v2}}}} rados supported-random-distro$/{centos_latest} thrashers/pggrow_host thrashosds-health workloads/redirect}
time:   00:14:00
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_11:21:23-rados:thrash-wip-osd-cluster-osdmap-lower-bound-by-last_epoch_started-distro-debug-trial/62685/
2026-02-21T13:23:47.864 ERROR:teuthology.results:error generating memo/results
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/results.py", line 33, in main
    results(args['--archive-dir'], args['--name'], args['--email'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/results.py", line 78, in results
    email_results(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/results.py", line 95, in email_results
    smtp = smtplib.SMTP('localhost')
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/smtplib.py", line 341, in connect
    self.sock = self._get_socket(host, port, self.timeout)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/smtplib.py", line 312, in _get_socket
    return socket.create_connection((host, port), timeout,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/socket.py", line 115, in create_connection
    sock.connect(sa)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_socketcommon.py", line 590, in connect
    self._internal_connect(address)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_socketcommon.py", line 634, in _internal_connect
    raise _SocketError(err, strerror(err))
ConnectionRefusedError: [Errno 111] Connection refused