Found 30 jobs Missing teuthology log /home/teuthworker/archive/gabrioux-2024-02-20_14:26:47-orch:cephadm-wip-guits-testing-6-2024-02-19-2119-pacific-distro-default-smithi/7568889/teuthology.log Found 5 distinct failure reasons Failure: "2024-02-20T16:11:31.679205+0000 mon.a (mon.0) 589 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log 17 jobs: ['7568912', '7568909', '7568898', '7568916', '7568893', '7568914', '7568899', '7568896', '7568906', '7568895', '7568900', '7568913', '7568905', '7568907', '7568887', '7568891', '7568902'] suites intersection: [] suites union: ['0-nvme-loop', '1-bootstrap/16.2.4', '1-bootstrap/16.2.5', '1-install/mimic-v1only', '1-install/octopus', '1-ranks/1', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '2-nfs', '2-ops/rmdir-reactivate}', '2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-inline/no', '3-inline/yes', '3-upgrade-with-workload', '3-upgrade/simple', '3-upgrade/staggered', '4-final}', '4-verify}', '4-verify}}', '4-wait', '5-upgrade-ls', 'backoff/peering', 'backoff/peering_and_degraded', 'centos_8.stream_container_tools', 'ceph', 'clusters/{openstack', 'conf/{client', 'd-balancer/crush-compat', 'd-balancer/on', 'distro$/{ubuntu_18.04}', 'fixed-2', 'ignorelist_wrongly_marked_down', 'mds', 'mode/packaged', 'mon', 'mon_election/classic', 'mon_election/classic}', 'mon_election/connectivity', 'mon_election/connectivity}', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async', 'msgr/async-v2only', 'orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'orch:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/osds/{0-distro/ubuntu_18.04', 'orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'orch:cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04', 'orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9', 'orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/with-work/{0-distro/ubuntu_18.04', 'orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'osd}', 'overrides/{ignorelist_health', 'pg-warn}', 'rados', 'roles', 'start', 'task/test_nfs}', 'task/test_orch_cli}', 'tasks/rados_api_tests}', 'tasks/{0-from/v16.2.4', 'thrashers/careful', 'thrashers/default', 'thrashers/pggrow', 'thrashosds-health', 'three-plus-one}', 'workloads/radosbench}', 'workloads/snaps-few-objects}'] Failure: expected string or bytes-like object 6 jobs: ['7568903', '7568915', '7568892', '7568908', '7568897', '7568888'] suites intersection: ['1-start', '2-thrash', 'fixed-2', 'orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools', 'root}'] suites union: ['1-start', '2-thrash', '3-tasks/rados_api_tests', '3-tasks/radosbench', '3-tasks/small-objects', '3-tasks/snaps-few-objects', 'fixed-2', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools', 'root}'] Dead: None 1 jobs: ['7568889'] Failure: Command failed on smithi157 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a23f2fac-d009-11ee-95bb-87774f69a715 -e sha1=19854089e18d4f65dda2b6cd74e737367c2514bd -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' 1 jobs: ['7568910'] suites: ['1-bootstrap/octopus', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'orch:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools'] Failure: Command failed on smithi115 with status 32: "sudo TESTDIR=/home/ubuntu/cephtest bash -ex -c 'mount -t nfs 10.0.31.115:/fake /mnt/foo -o port=2999'" 1 jobs: ['7568894'] suites: ['0-nvme-loop', '1-start', '2-services/nfs-ingress2', '3-final}', 'orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04']