Found 272 jobs
Found 14 distinct failure reasons
Crash: Command failed (workunit test cephtool/test_kvstore_tool.sh) on trial131 with status 139: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b5cebfcf1ca6de8a0e819c6217fece9f535154a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh'
ceph version 20.2.0-687-g7b5cebfc (7b5cebfcf1ca6de8a0e819c6217fece9f535154a) tentacle (stable - RelWithDebInfo)
 1: /lib64/libc.so.6(+0x3fc30) [0x7f8afe43fc30]
 2: (RocksDBStore::get_wholespace_iterator(unsigned int)+0xc7) [0x555d3c91c657]
 3: (RocksDBStore::get_iterator(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, KeyValueDB::IteratorBounds)+0x2d5) [0x555d3c91b435]
 4: (BlueStore::_close_db()+0x41a) [0x555d3c74491a]
 5: (BlueStore::umount()+0x1ac) [0x555d3c74e0dc]
 6: ceph-kvstore-tool(+0x235cc3) [0x555d3c6e8cc3]
 7: main()
 8: /lib64/libc.so.6(+0x2a610) [0x7f8afe42a610]
 9: __libc_start_main()
 10: _start()
7 jobs: ['63441', '63563', '63559', '63500', '63377', '63621', '63427']
suites intersection: ['rados']
suites union: ['base', 'mem$/{low}', 'mem$/{normal-1}', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr-failures/many', 'msgr-failures/none', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore/{alloc$/{avl}', 'objectstore/bluestore/{alloc$/{bitmap}', 'objectstore/bluestore/{alloc$/{btree}', 'objectstore/bluestore/{alloc$/{stupid}', 'onode-segment$/{256K}', 'onode-segment$/{512K-onoff}', 'onode-segment$/{512K}', 'onode-segment$/{none}', 'rados', 'rados/singleton-bluestore/{all/cephtool', 'rados/singleton-nomsgr/{all/ceph-kvstore-tool', 'random}}}', 'supported-random-distro$/{centos_latest}}', 'supported-random-distro$/{ubuntu_latest}}', 'v1}}}', 'v2}}}', 'write$/{random/{compr$/{no$/{no}}', 'write$/{v1/{compr$/{yes$/{snappy}}', 'write$/{v2/{compr$/{no$/{no}}', 'write$/{v2/{compr$/{yes$/{zstd}}']

Failure: "grep: /var/log/ceph/14496c7c-0fb0-11f1-9ebf-d404e6e7d460/ceph.log: No such file or directory" in cluster log
1 jobs: ['63405']
suites: ['1-tasks', 'rados/encoder/{0-start', 'supported-random-distro$/{centos_latest}}']

Failure: Test failure: test_list_enabled_module (tasks.mgr.dashboard.test_mgr_module.MgrModuleTest.test_list_enabled_module)
2 jobs: ['63404', '63543']
suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'tasks/dashboard}']
suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-zstd}', 'random-objectstore$/{bluestore-low-osd-mem-target}', 'tasks/dashboard}']

Dead: b'2026-02-22T12:00:11.728 DEBUG:teuthology.exit:Finished running handlers'
2 jobs: ['63499', '63542']
suites intersection: []
suites union: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-async-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-5}', 'agent/on', 'backoff/peering_and_degraded', 'base', 'ceph', 'clusters/{fixed-4}', 'crc-failures/bad_map_crc_failure', 'd-balancer/on', 'mem$/{normal-2}', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr/async', 'objectstore/{bluestore/{alloc$/{avl}', 'onode-segment$/{1M}', 'rados', 'rados/cephadm/workunits/{0-distro/centos_9.stream', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{centos_latest}', 'task/test_iscsi_container/{centos_9.stream', 'test_iscsi_container}}', 'thrashers/careful_host', 'thrashosds-health', 'v2}}}}', 'workloads/redirect_promote_tests}', 'write$/{v2/{compr$/{yes$/{zlib}}']

Failure: "2026-02-22T06:04:19.079539+0000 mon.c (mon.2) 281 : cluster [WRN] Health check failed: 1 network partition detected (MON_NETSPLIT)" in cluster log
4 jobs: ['63470', '63386', '63509', '63482']
suites intersection: []
suites union: ['0-size-min-size-overrides/3-size-2-min-size', '1-install/squid', '2-ceph', '3-mgrmodules', '4-units/module_selftest}}', 'backoff/normal', 'base', 'ceph', 'clusters/{three-plus-one}', 'd-balancer/on', 'debug/mgr', 'distro/{centos_latest}', 'mem$/{low}', 'mem$/{normal-1}', 'mgr_ttl_cache/enable', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/many', 'msgr-failures/osd-delay', 'msgr/async-v1only', 'msgr/async-v2only', 'no_pools', 'objectstore/{bluestore/{alloc$/{bitmap}', 'objectstore/{bluestore/{alloc$/{btree}', 'onode-segment$/{256K}', 'onode-segment$/{none}', 'rados', 'rados/mgr/{clusters/{2-node-mgr}', 'rados/multimon/{clusters/21', 'rados/singleton/{all/stretch-mode-5-mons-8-osds', 'rados/thrash-old-clients/{0-distro$/{centos_9.stream}', 'random-objectstore$/{bluestore/{alloc$/{btree}', 'random}}}}', 'supported-random-distro$/{ubuntu_latest}', 'supported-random-distro$/{ubuntu_latest}}', 'tasks/mon_recovery}', 'tasks/{1-install', 'thrashers/default', 'thrashosds-health', 'v1}}}}', 'workloads/radosbench}', 'write$/{random/{compr$/{yes$/{zlib}}', 'write$/{random/{compr$/{yes$/{zstd}}', 'write$/{v1/{compr$/{yes$/{zlib}}']

Failure: Command failed (workunit test post-file.sh) on trial116 with status 255: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b5cebfcf1ca6de8a0e819c6217fece9f535154a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/post-file.sh'
2 jobs: ['63432', '63570']
suites intersection: ['rados', 'rados/singleton-nomsgr/{all/ceph-post-file']
suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/ceph-post-file', 'supported-random-distro$/{centos_latest}}', 'supported-random-distro$/{ubuntu_latest}}']

Failure: Command failed (workunit test cephadm/test_cephadm.sh) on trial194 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b5cebfcf1ca6de8a0e819c6217fece9f535154a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'
2 jobs: ['63631', '63395']
suites intersection: ['agent/on', 'mon_election/connectivity', 'task/test_cephadm}']
suites union: ['agent/on', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/centos_9.stream', 'rados/cephadm/workunits/{0-distro/ubuntu_22.04', 'task/test_cephadm}']

Failure: Command failed (workunit test mon/misc.sh) on trial170 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b5cebfcf1ca6de8a0e819c6217fece9f535154a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/misc.sh'
1 jobs: ['63574']
suites: ['rados/standalone/{supported-random-distro$/{ubuntu_latest}', 'workloads/mon}']

Failure: ["Unexpected failure during module execution: [Errno 28] No space left on device: '/home/teuthworker/.ansible/tmp/ansible-local-2099321e0jggpzy/ansiballz_cache/ansible.modules.apt_key-ZIP_DEFLATED-part'"]
6 jobs: ['63464', '63453', '63472', '63462', '63457', '63458']
suites intersection: []
suites union: ['0-nvme-loop', '1-start', '2-ops/rm-zap-wait}', 'agent/off', 'base', 'ceph', 'clusters/fixed-4', 'clusters/{fixed-4}', 'd-thrash/none', 'ec_optimizations/ec_optimizations_off', 'mem$/{low}', 'mem$/{normal-2}', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/{bluestore/{alloc$/{avl}', 'objectstore/{bluestore/{alloc$/{stupid}', 'onode-segment$/{256K}', 'onode-segment$/{512K-onoff}', 'rados', 'rados/cephadm/osds/{0-distro/centos_9.stream', 'rados/cephadm/workunits/{0-distro/ubuntu_22.04', 'rados/objectstore/{backends/fusestore', 'rados/singleton-nomsgr/{all/health-warnings', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'rados/verify/{centos_latest', 'random}}}}', 'read-affinity/balance', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{centos_latest}}', 'supported-random-distro$/{ubuntu_latest}}', 'task/test_host_drain}', 'tasks/rados_api_tests', 'thrashers/none', 'thrashosds-health', 'v1}}}}', 'validater/valgrind}', 'workloads/ec-rados-plugin=isa-k=4-m=2}', 'write$/{random/{compr$/{yes$/{zstd}}', 'write$/{v1/{compr$/{no$/{no}}']

Failure: reached maximum tries (800) after waiting for 4800 seconds
1 jobs: ['63442']
suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-async-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-1}', 'backoff/normal', 'base', 'ceph', 'clusters/{fixed-4}', 'crc-failures/bad_map_crc_failure', 'd-balancer/on', 'mem$/{normal-1}', 'mon_election/classic', 'msgr-failures/fastclose', 'msgr/async-v1only', 'objectstore/{bluestore/{alloc$/{bitmap}', 'onode-segment$/{512K}', 'rados', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{centos_latest}', 'thrashers/default_host', 'thrashosds-health', 'v2}}}}', 'workloads/admin_socket_objecter_requests}', 'write$/{v2/{compr$/{no$/{no}}']

Failure: Command failed on trial020 with status 1: 'yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2'
2 jobs: ['63476', '63613']
suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'tasks/e2e}']
suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-snappy}', 'random-objectstore$/{bluestore-stupid}', 'tasks/e2e}']

Failure: Command failed with status 250: 'ansible-playbook -v --extra-vars \'{"ansible_ssh_user": "ubuntu"}\' -i /etc/ansible/hosts --limit trial122.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml'
1 jobs: ['63471']
suites: ['mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/large-omap-object-warnings', 'supported-random-distro$/{centos_latest}}']

Failure: Command failed (workunit test osd/osd-bluefs-volume-ops.sh) on trial162 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b5cebfcf1ca6de8a0e819c6217fece9f535154a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh'
1 jobs: ['63628']
suites: ['rados/standalone/{supported-random-distro$/{ubuntu_latest}', 'workloads/osd}']

Failure: Command failed (workunit test osd-backfill/osd-backfill-prio.sh) on trial104 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7b5cebfcf1ca6de8a0e819c6217fece9f535154a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd-backfill/osd-backfill-prio.sh'
1 jobs: ['63600']
suites: ['rados/standalone/{supported-random-distro$/{centos_latest}', 'workloads/osd-backfill}']