2026-02-21T09:48:34.533 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-21T09:48:34.533 INFO:teuthology.results:seed: '2477'
2026-02-21T09:49:23.615 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-21T09:49:23.615 INFO:teuthology.results:Waiting up to 7200 seconds for tests to finish...
2026-02-21T11:49:37.134 WARNING:teuthology.results:test(s) did not finish before timeout of 7200 seconds
2026-02-21T11:49:38.665 INFO:scrape:Found 71 jobs
2026-02-21T11:49:40.946 INFO:scrape:Found 9 distinct failure reasons
2026-02-21T11:49:40.946 INFO:scrape:Failure: Command failed (workunit test rbd/test_librbd_python.sh) on trial004 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd_python.sh'
2026-02-21T11:49:40.946 INFO:scrape:4 jobs: ['62349', '62325', '62368', '62321']
2026-02-21T11:49:40.947 INFO:scrape:suites intersection: ['clusters/crimson-fixed-1', 'crimson-rados/rbd/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph']
2026-02-21T11:49:40.947 INFO:scrape:suites union: ['clusters/crimson-fixed-1', 'crimson-rados/rbd/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/rbm$/{crimson_seastore_rbm_2q}', 'objectstore/seastore/rbm$/{crimson_seastore_rbm}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}', 'tasks/rbd_python_api_tests_old_format}', 'tasks/rbd_python_api_tests}']
2026-02-21T11:49:40.947 INFO:scrape:
2026-02-21T11:49:40.947 INFO:scrape:Failure: Command failed (workunit test cephtool/test_crimson_dump_metrics.sh) on trial122 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_crimson_dump_metrics.sh'
2026-02-21T11:49:40.947 INFO:scrape:2 jobs: ['62314', '62344']
2026-02-21T11:49:40.947 INFO:scrape:suites intersection: ['all/osd-dump-metrics', 'crimson-rados/singleton/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'rados}']
2026-02-21T11:49:40.947 INFO:scrape:suites union: ['all/osd-dump-metrics', 'crimson-rados/singleton/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'objectstore/seastore/rbm$/{crimson_seastore_rbm}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}', 'rados}']
2026-02-21T11:49:40.947 INFO:scrape:
2026-02-21T11:49:40.947 INFO:scrape:Failure: "2026-02-21T10:25:04.860278+0000 mon.a (mon.0) 182 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log
2026-02-21T11:49:40.947 INFO:scrape:1 jobs: ['62371']
2026-02-21T11:49:40.947 INFO:scrape:suites: ['clusters/crimson-fixed-2', 'crimson-rados/basic/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'tasks/readwrite}']
2026-02-21T11:49:40.947 INFO:scrape:
2026-02-21T11:49:40.947 INFO:scrape:Timeout 120 running ceph --cluster ceph osd dump --format=json
2026-02-21T11:49:40.947 INFO:scrape:2 jobs: ['62375', '62350']
2026-02-21T11:49:40.947 INFO:scrape:suites intersection: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados/thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'thrashers/{pggrow}', 'thrashosds-health']
2026-02-21T11:49:40.947 INFO:scrape:suites union: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados/thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'thrashers/{pggrow}', 'thrashosds-health', 'workloads/snaps-few-objects-localized}', 'workloads/write_fadvise_dontneed}']
2026-02-21T11:49:40.947 INFO:scrape:
2026-02-21T11:49:40.947 INFO:scrape:Failure: Command failed on trial052 with status 6: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats'
2026-02-21T11:49:40.947 INFO:scrape:1 jobs: ['62340']
2026-02-21T11:49:40.947 INFO:scrape:suites: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados/thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'thrashers/{default}', 'thrashosds-health', 'workloads/write_fadvise_dontneed}']
2026-02-21T11:49:40.947 INFO:scrape:
2026-02-21T11:49:40.947 INFO:scrape:Failure: The WatchDog barked due to Thrasher ceph.osd.1 threw exception timed out waiting for admin_socket to appear after osd.1 restart
2026-02-21T11:49:40.947 INFO:scrape:1 jobs: ['62362']
2026-02-21T11:49:40.947 INFO:scrape:suites: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados/thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'thrashers/{default}', 'thrashosds-health', 'workloads/snaps-few-objects}']
2026-02-21T11:49:40.947 INFO:scrape:
2026-02-21T11:49:40.947 INFO:scrape:Dead: b'2026-02-21T10:34:35.959 INFO:tasks.workunit.client.0.trial148.stdout:created completion'
2026-02-21T11:49:40.948 INFO:scrape:1 jobs: ['62357']
2026-02-21T11:49:40.948 INFO:scrape:suites: ['clusters/crimson-fixed-1', 'crimson-rados/rbd/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}', 'tasks/rbd_api_tests_old_format}']
2026-02-21T11:49:40.948 INFO:scrape:
2026-02-21T11:49:40.948 INFO:scrape:Failure: The WatchDog barked due to Daemon ceph.osd.2 has failed
2026-02-21T11:49:40.948 INFO:scrape:1 jobs: ['62366']
2026-02-21T11:49:40.948 INFO:scrape:suites: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados/thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'thrashers/{default}', 'thrashosds-health', 'workloads/pool-snaps-few-objects}']
2026-02-21T11:49:40.948 INFO:scrape:
2026-02-21T11:49:40.948 INFO:scrape:Dead: b'2026-02-21T10:37:42.415 INFO:tasks.ceph.mgr.x.trial020.stderr:2026-02-21T10:37:42.414+0000 7f8e397f8640 -1 mgr handle_mgr_signal  *** Got signal Terminated ***'
2026-02-21T11:49:40.948 INFO:scrape:1 jobs: ['62329']
2026-02-21T11:49:40.948 INFO:scrape:suites: ['clusters/crimson-fixed-1', 'crimson-rados/rbd/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}', 'tasks/rbd_api_tests}']
2026-02-21T11:49:40.948 INFO:scrape:
2026-02-21T11:49:40.949 INFO:teuthology.results:Sending results to xxhdx1985126@gmail.com: Test Run: xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial
=================================================================
info:    https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/
logs:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/
failed:  12
dead:    0
running: 2
waiting: 0
queued:  0
passed:  57


Fail
=================================================================
[62314]  crimson-rados/singleton/{0-crimson_install all/osd-dump-metrics crimson-supported-all-distro/centos_latest crimson_qa_overrides objectstore/seastore/rbm$/{crimson_seastore_rbm} rados}
-----------------------------------------------------------------
time:   00:30:50
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62314/
log:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62314/

    Command failed (workunit test cephtool/test_crimson_dump_metrics.sh) on
    trial122 with status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cep
    htest/clone.client.0/qa/workunits/cephtool/test_crimson_dump_metrics.sh'

[62321]  crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} tasks/rbd_python_api_tests}
-----------------------------------------------------------------
time:   00:24:35
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62321/
log:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62321/

    Command failed (workunit test rbd/test_librbd_python.sh) on trial118 with
    status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 CRIMSON_COMPAT=1
    adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout
    3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd_python
    .sh'

[62325]  crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} tasks/rbd_python_api_tests_old_format}
-----------------------------------------------------------------
time:   00:18:51
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62325/
log:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62325/

    Command failed (workunit test rbd/test_librbd_python.sh) on trial184 with
    status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cep
    htest/clone.client.0/qa/workunits/rbd/test_librbd_python.sh'

[62340]  crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{default} thrashosds-health workloads/write_fadvise_dontneed}
-----------------------------------------------------------------
time:   00:48:08
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62340/
log:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62340/

    Command failed on trial052 with status 6: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph tell osd.2 flush_pg_stats'

[62344]  crimson-rados/singleton/{0-crimson_install all/osd-dump-metrics crimson-supported-all-distro/centos_latest crimson_qa_overrides objectstore/seastore/segmented$/{crimson_seastore_segmented} rados}
-----------------------------------------------------------------
time:   00:28:30
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62344/
log:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62344/

    Command failed (workunit test cephtool/test_crimson_dump_metrics.sh) on
    trial145 with status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cep
    htest/clone.client.0/qa/workunits/cephtool/test_crimson_dump_metrics.sh'

[62349]  crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} tasks/rbd_python_api_tests_old_format}
-----------------------------------------------------------------
time:   00:19:04
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62349/
log:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62349/

    Command failed (workunit test rbd/test_librbd_python.sh) on trial004 with
    status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cep
    htest/clone.client.0/qa/workunits/rbd/test_librbd_python.sh'

[62350]  crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{pggrow} thrashosds-health workloads/snaps-few-objects-localized}
-----------------------------------------------------------------
time:   00:53:41
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62350/
log:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62350/

    Command failed on trial042 with status 124: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph osd dump --format=json'

[62362]  crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{default} thrashosds-health workloads/snaps-few-objects}
-----------------------------------------------------------------
time:   00:49:57
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62362/
log:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62362/

    The WatchDog barked due to Thrasher ceph.osd.1 threw exception timed out
    waiting for admin_socket to appear after osd.1 restart

[62366]  crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{default} thrashosds-health workloads/pool-snaps-few-objects}
-----------------------------------------------------------------
time:   01:05:44
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62366/
log:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62366/

    The WatchDog barked due to Daemon ceph.osd.2 has failed

[62368]  crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/rbd_python_api_tests}
-----------------------------------------------------------------
time:   00:26:07
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62368/
log:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62368/

    Command failed (workunit test rbd/test_librbd_python.sh) on trial028 with
    status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 CRIMSON_COMPAT=1
    adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout
    3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd_python
    .sh'

[62371]  crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/readwrite}
-----------------------------------------------------------------
time:   00:38:25
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62371/
log:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62371/

    "2026-02-21T10:25:04.860278+0000 mon.a (mon.0) 182 : cluster [WRN] Health
    check failed: 1 osds down (OSD_DOWN)" in cluster log

[62375]  crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{pggrow} thrashosds-health workloads/write_fadvise_dontneed}
-----------------------------------------------------------------
time:   00:42:56
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62375/
log:    https://qa-proxy.ceph.com/teuthology/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62375/

    Command failed on trial043 with status 124: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph osd dump --format=json'



Running
=================================================================
[62329] crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} tasks/rbd_api_tests}
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62329/

[62357] crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} tasks/rbd_api_tests_old_format}
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62357/



Pass
=================================================================
[62305] crimson-rados/objectstore_tool/{0-crimson_install clusters/fixed-1-num-threads-3 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/seastore-segmented tasks/crimson_obejctstore_tool}
time:   00:29:06
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62305/

[62306] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} thrashers/{pggrow} thrashosds-health workloads/pool-snaps-few-objects}
time:   00:51:49
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62306/

[62307] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} thrashers/{default} thrashosds-health workloads/radosbench-high-concurrency}
time:   00:50:25
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62307/

[62308] crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} tasks/rbd_api_tests_old_format}
time:   00:53:38
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62308/

[62309] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{default} thrashosds-health workloads/small-objects-balanced}
time:   00:40:20
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62309/

[62310] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} thrashers/{pggrow} thrashosds-health workloads/small-objects-localized}
time:   00:42:36
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62310/

[62311] crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} tasks/rados_api_tests}
time:   00:45:19
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62311/

[62312] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{default} thrashosds-health workloads/snaps-few-objects-balanced}
time:   01:24:17
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62312/

[62313] crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} tasks/rbd_cls_tests}
time:   00:30:14
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62313/

[62315] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{pggrow} thrashosds-health workloads/snaps-few-objects-localized}
time:   00:49:35
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62315/

[62316] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} thrashers/{pggrow} thrashosds-health workloads/write_fadvise_dontneed}
time:   00:29:48
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62316/

[62317] crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} tasks/rados_python}
time:   00:29:42
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62317/

[62318] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{pggrow} thrashosds-health workloads/admin_socket_objecter_requests}
time:   00:36:38
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62318/

[62319] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} thrashers/{default} thrashosds-health workloads/radosbench-high-concurrency}
time:   00:51:18
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62319/

[62320] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{pggrow} thrashosds-health workloads/radosbench}
time:   00:45:47
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62320/

[62322] crimson-rados/singleton/{0-crimson_install all/osd-pg-splitting crimson-supported-all-distro/centos_latest crimson_qa_overrides objectstore/seastore/segmented$/{crimson_seastore_segmented} rados}
time:   00:33:07
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62322/

[62323] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} thrashers/{pggrow} thrashosds-health workloads/small-objects-localized}
time:   00:38:39
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62323/

[62324] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} thrashers/{pggrow} thrashosds-health workloads/small-objects}
time:   00:44:28
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62324/

[62326] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} thrashers/{default} thrashosds-health workloads/snaps-few-objects-localized}
time:   01:19:37
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62326/

[62327] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{pggrow} thrashosds-health workloads/snaps-few-objects}
time:   00:50:03
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62327/

[62328] crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} tasks/cls}
time:   00:28:23
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62328/

[62330] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} thrashers/{pggrow} thrashosds-health workloads/admin_socket_objecter_requests}
time:   00:36:21
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62330/

[62331] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} thrashers/{pggrow} thrashosds-health workloads/pool-snaps-few-objects}
time:   00:50:39
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62331/

[62332] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{default} thrashosds-health workloads/radosbench}
time:   00:48:00
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62332/

[62333] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{pggrow} thrashosds-health workloads/small-objects-balanced}
time:   00:40:45
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62333/

[62334] crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} tasks/rados_api_tests}
time:   00:50:17
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62334/

[62335] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} thrashers/{pggrow} thrashosds-health workloads/small-objects}
time:   00:41:35
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62335/

[62336] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{pggrow} thrashosds-health workloads/snaps-few-objects-balanced}
time:   00:45:53
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62336/

[62337] crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} tasks/rbd_cls_tests}
time:   00:33:12
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62337/

[62338] crimson-rados/singleton/{0-crimson_install all/osd-backfill crimson-supported-all-distro/centos_latest crimson_qa_overrides objectstore/seastore/rbm$/{crimson_seastore_rbm} rados}
time:   00:32:17
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62338/

[62339] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{pggrow} thrashosds-health workloads/snaps-few-objects}
time:   00:47:47
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62339/

[62341] crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} tasks/rbd_lock_and_fence}
time:   00:28:36
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62341/

[62342] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} thrashers/{default} thrashosds-health workloads/pool-snaps-few-objects}
time:   01:33:39
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62342/

[62343] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{default} thrashosds-health workloads/radosbench-high-concurrency}
time:   00:49:36
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62343/

[62345] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} thrashers/{pggrow} thrashosds-health workloads/small-objects-balanced}
time:   00:41:16
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62345/

[62346] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} thrashers/{default} thrashosds-health workloads/small-objects-localized}
time:   00:40:05
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62346/

[62347] crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} tasks/readwrite}
time:   00:33:35
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62347/

[62348] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{default} thrashosds-health workloads/snaps-few-objects-balanced}
time:   01:20:01
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62348/

[62351] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{pggrow} thrashosds-health workloads/write_fadvise_dontneed}
time:   00:33:20
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62351/

[62352] crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/cls}
time:   00:29:42
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62352/

[62353] crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} tasks/rbd_api_tests}
time:   01:35:38
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62353/

[62354] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} thrashers/{pggrow} thrashosds-health workloads/admin_socket_objecter_requests}
time:   00:36:42
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62354/

[62355] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} thrashers/{default} thrashosds-health workloads/radosbench-high-concurrency}
time:   00:47:15
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62355/

[62356] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} thrashers/{pggrow} thrashosds-health workloads/radosbench}
time:   00:49:10
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62356/

[62358] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{pggrow} thrashosds-health workloads/small-objects-localized}
time:   00:40:15
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62358/

[62359] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{pggrow} thrashosds-health workloads/small-objects}
time:   00:43:39
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62359/

[62360] crimson-rados/singleton/{0-crimson_install all/osd-pg-splitting crimson-supported-all-distro/centos_latest crimson_qa_overrides objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} rados}
time:   00:32:08
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62360/

[62361] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{pggrow} thrashosds-health workloads/snaps-few-objects-localized}
time:   00:47:06
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62361/

[62363] crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} tasks/rados_python}
time:   00:32:02
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62363/

[62364] crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} tasks/rbd_lock_and_fence}
time:   00:28:43
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62364/

[62365] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{pggrow} thrashosds-health workloads/admin_socket_objecter_requests}
time:   00:32:55
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62365/

[62367] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{default} thrashosds-health workloads/radosbench}
time:   00:46:52
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62367/

[62369] crimson-rados/singleton/{0-crimson_install all/osd-backfill crimson-supported-all-distro/centos_latest crimson_qa_overrides objectstore/seastore/segmented$/{crimson_seastore_segmented} rados}
time:   00:29:38
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62369/

[62370] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} thrashers/{default} thrashosds-health workloads/small-objects-balanced}
time:   00:50:44
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62370/

[62372] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{pggrow} thrashosds-health workloads/small-objects}
time:   00:41:00
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62372/

[62373] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} thrashers/{pggrow} thrashosds-health workloads/snaps-few-objects-balanced}
time:   00:48:22
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62373/

[62374] crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} thrashers/{pggrow} thrashosds-health workloads/snaps-few-objects}
time:   00:48:16
info:   https://pulpito.ceph.com/xuxuehan-2026-02-21_09:48:09-crimson-rados-wip-trans-cc-opt-rbm-drop-brf-distro-debug-trial/62374/
2026-02-21T11:49:40.958 ERROR:teuthology.results:error generating memo/results
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/results.py", line 33, in main
    results(args['--archive-dir'], args['--name'], args['--email'],
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/results.py", line 78, in results
    email_results(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/results.py", line 95, in email_results
    smtp = smtplib.SMTP('localhost')
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/smtplib.py", line 341, in connect
    self.sock = self._get_socket(host, port, self.timeout)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/smtplib.py", line 312, in _get_socket
    return socket.create_connection((host, port), timeout,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/socket.py", line 115, in create_connection
    sock.connect(sa)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_socketcommon.py", line 590, in connect
    self._internal_connect(address)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_socketcommon.py", line 634, in _internal_connect
    raise _SocketError(err, strerror(err))
ConnectionRefusedError: [Errno 111] Connection refused