Found 74 jobs
Missing teuthology log /home/teuthworker/archive/teuthology-2024-02-08_02:25:06-upgrade:reef-x-main-distro-default-smithi/7551518/teuthology.log
Found 8 distinct failure reasons
Failure: Command failed on smithi071 with status 1: 'sudo yum install -y kernel'
29 jobs: ['7551480', '7551502', '7551491', '7551475', '7551493', '7551487', '7551484', '7551496', '7551499', '7551490', '7551512', '7551476', '7551470', '7551527', '7551488', '7551472', '7551514', '7551483', '7551477', '7551479', '7551473', '7551494', '7551495', '7551528', '7551530', '7551520', '7551524', '7551498', '7551503']
suites intersection: []
suites union: ['0-roles', '0-start', '1-start', '1-tasks', '2-first-half-tasks/radosbench', '2-first-half-tasks/rbd-cls', '2-first-half-tasks/rbd-import-export', '2-first-half-tasks/rbd_api', '2-first-half-tasks/readwrite', '2-first-half-tasks/snaps-few-objects', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic}', 'mon_election/connectivity', 'mon_election/connectivity}', 'rados_api', 'rados_loadgenbig', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'rbd_import_export', 'readwrite', 'snaps-few-objects}', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'upgrade:reef-x/parallel/{0-random-distro$/{centos_9.stream_runc}', 'upgrade:reef-x/stress-split/{0-distro/centos_9.stream', 'upgrade:reef-x/stress-split/{0-distro/centos_9.stream_runc', 'workload/{ec-rados-default']

Failure: timeout expired in wait_until_healthy
3 jobs: ['7551469', '7551468', '7551463']
suites intersection: ['0-roles', '1-start', '3-stress-tasks/{radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}']
suites union: ['0-roles', '1-start', '2-first-half-tasks/readwrite', '2-first-half-tasks/snaps-few-objects', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic}', 'mon_election/connectivity}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:reef-x/stress-split/{0-distro/centos_9.stream', 'upgrade:reef-x/stress-split/{0-distro/centos_9.stream_runc', 'upgrade:reef-x/stress-split/{0-distro/ubuntu_22.04']

Failure: ["Failed to download remote objects and refs:  fatal: unable to access 'https://github.com/ceph/keys/': Could not resolve host: github.com"]
13 jobs: ['7551526', '7551522', '7551481', '7551474', '7551529', '7551500', '7551489', '7551497', '7551501', '7551478', '7551482', '7551492', '7551504']
suites intersection: ['0-roles', '1-start', '3-stress-tasks/{radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:reef-x/stress-split/{0-distro/ubuntu_22.04']
suites union: ['0-roles', '1-start', '2-first-half-tasks/radosbench', '2-first-half-tasks/rbd-cls', '2-first-half-tasks/rbd-import-export', '2-first-half-tasks/rbd_api', '2-first-half-tasks/readwrite', '2-first-half-tasks/snaps-few-objects', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic}', 'mon_election/connectivity}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:reef-x/stress-split/{0-distro/ubuntu_22.04']

Failure: "2024-02-08T05:13:25.429459+0000 mon.a (mon.0) 181 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log
5 jobs: ['7551467', '7551457', '7551458', '7551464', '7551459']
suites intersection: []
suites union: ['0-roles', '0-start', '1-start', '1-tasks', '2-first-half-tasks/radosbench', '2-first-half-tasks/rbd-cls', '2-first-half-tasks/rbd_api', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic', 'mon_election/classic}', 'mon_election/connectivity}', 'rados_api', 'rados_loadgenbig', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'rbd_import_export', 'readwrite', 'snaps-few-objects}', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'upgrade:reef-x/parallel/{0-random-distro$/{centos_9.stream}', 'upgrade:reef-x/stress-split/{0-distro/centos_9.stream', 'upgrade:reef-x/stress-split/{0-distro/centos_9.stream_runc', 'upgrade:reef-x/stress-split/{0-distro/ubuntu_22.04', 'workload/{ec-rados-default']

Dead: None
17 jobs: ['7551518', '7551517', '7551485', '7551511', '7551508', '7551507', '7551519', '7551515', '7551510', '7551505', '7551509', '7551513', '7551516', '7551523', '7551506', '7551525', '7551521']

Dead: b'2024-02-08T14:53:22.303 DEBUG:teuthology.exit:Finished running handlers'
3 jobs: ['7551460', '7551462', '7551471']
suites intersection: ['0-roles', '1-start', '3-stress-tasks/{radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}']
suites union: ['0-roles', '1-start', '2-first-half-tasks/rbd-cls', '2-first-half-tasks/rbd-import-export', '2-first-half-tasks/readwrite', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic}', 'mon_election/connectivity}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:reef-x/stress-split/{0-distro/centos_9.stream_runc', 'upgrade:reef-x/stress-split/{0-distro/ubuntu_22.04']

Failure: Command failed on smithi132 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone --depth 1 --branch reef https://github.com/ceph/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0'
3 jobs: ['7551461', '7551466', '7551465']
suites intersection: ['0-roles', '1-start', '3-stress-tasks/{radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}']
suites union: ['0-roles', '1-start', '2-first-half-tasks/rbd-cls', '2-first-half-tasks/rbd-import-export', '2-first-half-tasks/rbd_api', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic}', 'mon_election/connectivity}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:reef-x/stress-split/{0-distro/centos_9.stream', 'upgrade:reef-x/stress-split/{0-distro/centos_9.stream_runc']

Failure: machine smithi125.front.sepia.ceph.com is not locked
1 jobs: ['7551486']
suites: ['0-roles', '1-start', '2-first-half-tasks/readwrite', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', 'mon_election/connectivity}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:reef-x/stress-split/{0-distro/centos_9.stream']