Found 10 jobs Found 5 distinct failure reasons Dead: b'2023-12-16T12:57:28.230 DEBUG:teuthology.exit:Finished running handlers' 6 jobs: ['7493418', '7493431', '7493426', '7493428', '7493424', '7493422'] suites intersection: [] suites union: ['0-roles', '0-start', '1-start', '1-tasks', '2-first-half-tasks/radosbench', '2-first-half-tasks/rbd-cls', '2-first-half-tasks/rbd-import-export', '2-first-half-tasks/rbd_api', '2-first-half-tasks/readwrite', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic}', 'mon_election/connectivity', 'mon_election/connectivity}', 'rados_api', 'rados_loadgenbig', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'rbd_import_export', 'readwrite', 'snaps-few-objects}', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'upgrade:quincy-x/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0}', 'upgrade:quincy-x/stress-split/{0-distro/centos_8.stream_container_tools', 'upgrade:quincy-x/stress-split/{0-distro/centos_8.stream_container_tools_crun', 'upgrade:quincy-x/stress-split/{0-distro/rhel_8.6_container_tools_3.0', 'upgrade:quincy-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8', 'workload/{ec-rados-default'] Dead: b'2023-12-16T13:11:26.401 INFO:journalctl@ceph.mon.c.smithi123.stdout:Dec 16 13:11:26 smithi123 bash[103934]: cluster 2023-12-16T13:11:24.597152+0000 mgr.y (mgr.25075) 22925 : cluster 0 pgmap v22793: 103 pgs: 103 active+clean; 8.6 MiB data, 2.6 GiB used, 713 GiB / 715 GiB avail' 1 jobs: ['7493437'] suites: ['0-roles', '1-start', '2-first-half-tasks/readwrite', '3-stress-tasks/{radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:quincy-x/stress-split/{0-distro/ubuntu_20.04'] Failure: "1702689862.8637304 mon.a (mon.0) 60 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log 1 jobs: ['7493433'] suites: ['-', '1-ceph-install/quincy', '2', 'objectstore/filestore-xfs', 'start}', 'ubuntu_20.04}', 'upgrade', 'upgrade:quincy-x/filestore-remove-check/{0-cluster/{openstack'] Dead: b'2023-12-16T13:11:25.980 INFO:journalctl@ceph.mon.c.smithi043.stdout:Dec 16 13:11:25 smithi043 ceph-mon[115054]: from=\'mgr.25091 172.21.15.43:0/1845062785\' entity=\'mgr.y\' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch' 1 jobs: ['7493435'] suites: ['0-start', '1-tasks', 'mon_election/classic', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api', 'test_rbd_python}}', 'upgrade-sequence', 'upgrade:quincy-x/parallel/{0-random-distro$/{rhel_8.6_container_tools_rhel8}', 'workload/{ec-rados-default'] Failure: timeout expired in wait_until_healthy 1 jobs: ['7493420'] suites: ['0-roles', '1-start', '2-first-half-tasks/snaps-few-objects', '3-stress-tasks/{radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:quincy-x/stress-split/{0-distro/ubuntu_20.04']