2026-02-20T11:03:07.885 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-20T11:03:07.885 INFO:teuthology.results:seed: '1736'
2026-02-20T11:56:26.797 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-20T11:56:26.804 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2026-02-20T18:00:10.284 INFO:teuthology.results:Tests finished! gathering results...
2026-02-20T18:00:13.481 INFO:scrape:Found 145 jobs
2026-02-20T18:00:16.863 INFO:scrape:Found 8 distinct failure reasons
2026-02-20T18:00:16.863 INFO:scrape:Dead: b'2026-02-20T17:33:11.698 DEBUG:teuthology.exit:Finished running handlers'
2026-02-20T18:00:16.863 INFO:scrape:12 jobs: ['60548', '60503', '60556', '60599', '60589', '60544', '60592', '60500', '60566', '60614', '60604', '60493']
2026-02-20T18:00:16.863 INFO:scrape:suites intersection: ['clusters/{crimson-fixed-1}', 'crimson-rados/perf/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'perf_ignorelist', 'settings/optimized']
2026-02-20T18:00:16.863 INFO:scrape:suites union: ['clusters/{crimson-fixed-1}', 'crimson-rados/perf/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/rbm$/{crimson_seastore_rbm_2q}', 'objectstore/seastore/rbm$/{crimson_seastore_rbm}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}', 'perf_ignorelist', 'settings/optimized', 'workloads/radosbench_4K_rand_read}', 'workloads/radosbench_4K_seq_read}', 'workloads/radosbench_4M_rand_read}', 'workloads/radosbench_4M_seq_read}', 'workloads/radosbench_4M_write}', 'workloads/sample_radosbench}']
2026-02-20T18:00:16.863 INFO:scrape:
2026-02-20T18:00:16.863 INFO:scrape:Failure: The WatchDog barked due to Daemon ceph.osd.1 has failed
2026-02-20T18:00:16.863 INFO:scrape:27 jobs: ['60567', '60542', '60492', '60580', '60489', '60531', '60609', '60613', '60495', '60590', '60564', '60557', '60588', '60512', '60535', '60608', '60543', '60517', '60483', '60555', '60484', '60549', '60477', '60597', '60509', '60561', '60583']
2026-02-20T18:00:16.863 INFO:scrape:suites intersection: ['0-size-min-size-overrides/3-size-2-min-size', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados/thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'thrashosds-health']
2026-02-20T18:00:16.864 INFO:scrape:suites union: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados/thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/rbm$/{crimson_seastore_rbm_2q}', 'objectstore/seastore/rbm$/{crimson_seastore_rbm}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}', 'thrashers/{default}', 'thrashers/{pggrow}', 'thrashosds-health', 'workloads/pool-snaps-few-objects}', 'workloads/small-objects-balanced}', 'workloads/small-objects-localized}', 'workloads/small-objects}', 'workloads/snaps-few-objects-balanced}', 'workloads/snaps-few-objects-localized}', 'workloads/snaps-few-objects}', 'workloads/write_fadvise_dontneed}']
2026-02-20T18:00:16.864 INFO:scrape:
2026-02-20T18:00:16.864 INFO:scrape:Failure: reached maximum tries (500) after waiting for 3000 seconds
2026-02-20T18:00:16.864 INFO:scrape:7 jobs: ['60479', '60595', '60499', '60504', '60528', '60572', '60524']
2026-02-20T18:00:16.864 INFO:scrape:suites intersection: ['0-size-min-size-overrides/3-size-2-min-size', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados/thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'thrashosds-health']
2026-02-20T18:00:16.864 INFO:scrape:suites union: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados/thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/rbm$/{crimson_seastore_rbm}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}', 'thrashers/{default}', 'thrashers/{pggrow}', 'thrashosds-health', 'workloads/admin_socket_objecter_requests}', 'workloads/radosbench-high-concurrency}', 'workloads/radosbench}']
2026-02-20T18:00:16.864 INFO:scrape:
2026-02-20T18:00:16.864 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/test_librbd.sh
2026-02-20T18:00:16.864 INFO:scrape:4 jobs: ['60578', '60570', '60522', '60482']
2026-02-20T18:00:16.864 INFO:scrape:suites intersection: ['clusters/crimson-fixed-1', 'crimson-rados/rbd/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph']
2026-02-20T18:00:16.864 INFO:scrape:suites union: ['clusters/crimson-fixed-1', 'crimson-rados/rbd/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/rbm$/{crimson_seastore_rbm_2q}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}', 'tasks/rbd_api_tests_old_format}', 'tasks/rbd_api_tests}']
2026-02-20T18:00:16.864 INFO:scrape:
2026-02-20T18:00:16.864 INFO:scrape:Timeout 120 running ceph --cluster ceph pg dump --format=json
2026-02-20T18:00:16.864 INFO:scrape:2 jobs: ['60539', '60602']
2026-02-20T18:00:16.864 INFO:scrape:suites intersection: ['all/osd-backfill', 'crimson-rados/singleton/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'rados}']
2026-02-20T18:00:16.864 INFO:scrape:suites union: ['all/osd-backfill', 'crimson-rados/singleton/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'objectstore/seastore/rbm$/{crimson_seastore_rbm_2q}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'rados}']
2026-02-20T18:00:16.864 INFO:scrape:
2026-02-20T18:00:16.864 INFO:scrape:Failure: timed out waiting for admin_socket to appear after osd.0 restart
2026-02-20T18:00:16.864 INFO:scrape:1 jobs: ['60472']
2026-02-20T18:00:16.864 INFO:scrape:suites: ['clusters/fixed-1-num-threads-3', 'crimson-rados/objectstore_tool/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/seastore-segmented', 'tasks/crimson_obejctstore_tool}']
2026-02-20T18:00:16.864 INFO:scrape:
2026-02-20T18:00:16.864 INFO:scrape:Failure: Command failed (workunit test rados/test_pg_splitting.sh) on trial118 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=445d5c9ba18a4040e70e25303230c36c77773acb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pg_splitting.sh'
2026-02-20T18:00:16.864 INFO:scrape:2 jobs: ['60506', '60587']
2026-02-20T18:00:16.864 INFO:scrape:suites intersection: ['all/osd-pg-splitting', 'crimson-rados/singleton/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'rados}']
2026-02-20T18:00:16.864 INFO:scrape:suites union: ['all/osd-pg-splitting', 'crimson-rados/singleton/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'objectstore/seastore/rbm$/{crimson_seastore_rbm}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'rados}']
2026-02-20T18:00:16.864 INFO:scrape:
2026-02-20T18:00:16.864 INFO:scrape:Timeout 120 running ceph --cluster ceph osd dump --format=json
2026-02-20T18:00:16.865 INFO:scrape:1 jobs: ['60615']
2026-02-20T18:00:16.865 INFO:scrape:suites: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados/thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}', 'thrashers/{default}', 'thrashosds-health', 'workloads/write_fadvise_dontneed}']
2026-02-20T18:00:16.865 INFO:scrape: