Found 44 jobs Missing teuthology log /home/teuthworker/archive/yuriw-2023-07-15_23:21:32-orch-reef-distro-default-smithi/7340525/teuthology.log Found 6 distinct failure reasons Failure: Test failure: test_update_export_ro_to_rw (tasks.cephfs.test_nfs.TestNFS) 1 jobs: ['7340511'] suites: ['agent/on', 'mon_election/connectivity', 'orch/cephadm/workunits/{0-distro/ubuntu_20.04', 'task/test_nfs}'] Failure: Command failed on smithi120 with status 22: "sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32d8d82c-2378-11ee-9b33-001a4aab830c -- bash -c 'ceph orch upgrade ls'" 6 jobs: ['7340513', '7340568', '7340543', '7340591', '7340508', '7340533'] suites intersection: ['4-wait', '5-upgrade-ls'] suites union: ['2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-upgrade/simple', '3-upgrade/staggered', '4-wait', '5-upgrade-ls', 'agent/off', 'agent/on', 'mon_election/classic}', 'mon_election/connectivity}', 'orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools', 'orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04'] Failure: Command failed on smithi066 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" 4 jobs: ['7340546', '7340577', '7340527', '7340516'] suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', 'k8s/1.21', 'orch/rook/smoke/{0-distro/ubuntu_20.04'] suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '2-workload/radosbench', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'net/flannel', 'net/host', 'orch/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}', 'rook/master}'] Failure: Stale jobs detected, aborting. 1 jobs: ['7340523'] suites: ['2-repo_digest/defaut', '3-upgrade/staggered', '4-wait', '5-upgrade-ls', 'agent/off', 'mon_election/classic}', 'orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools'] Dead: None 2 jobs: ['7340525', '7340521'] Failure: reached maximum tries (301) after waiting for 300 seconds 2 jobs: ['7340517', '7340541'] suites intersection: ['0-nvme-loop', '1-start', '2-services/jaeger', '3-final}'] suites union: ['0-nvme-loop', '1-start', '2-services/jaeger', '3-final}', 'orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun', 'orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0']