Found 4 jobs Missing teuthology log /home/teuthworker/archive/xiubli-2023-03-17_01:16:58-fs:workload-wip-lxb-20230316-2212-unlink-revert-distro-default-smithi/7211345/teuthology.log Found 3 distinct failure reasons Dead: None 1 jobs: ['7211345'] Dead: b'2023-03-17T15:36:45.883 DEBUG:teuthology.exit:Finished running handlers' 2 jobs: ['7211343', '7211344'] suites intersection: ['1-cephadm', '1-check-counter', '2-logrotate}', '5-workunit/suites/pjd}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs:workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'ranks/1', 'rhel_8}', 'session_timeout}', 'standby-replay', 'wsync/yes}'] suites union: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/no', '2-scrub/yes', '3-snaps/no', '3-snaps/yes', '4-flush/no', '4-flush/yes', '5-workunit/suites/pjd}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs:workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'ms_mode/crc', 'ms_mode/legacy', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'ranks/1', 'rhel_8}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-namespace-isolated-and-quota}', 'tasks/{0-subvolume/{with-quota}', 'wsync/yes}'] Dead: b'2023-03-17T17:44:57.856 INFO:journalctl@ceph.osd.4.smithi093.stdout:Mar 17 17:32:44 smithi093 ceph-b9a42346-c489-11ed-9afb-001a4aab830c-osd-4[105352]: 2023-03-17T17:32:44.533+0000 7f8bcf7ce700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 ever on either front or back, first ping sent 2023-03-17T17:31:32.062893+0000 (oldest deadline 2023-03-17T17:31:52.062893+0000)' 1 jobs: ['7211346'] suites: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/no', '3-snaps/no', '4-flush/yes', '5-workunit/suites/pjd}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs:workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/crc', 'objectstore-ec/bluestore-bitmap', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'ranks/1', 'rhel_8}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-namespace-isolated-and-quota}', 'wsync/no}']