Found 94 jobs Missing teuthology log /home/teuthworker/archive/adking-2024-02-01_02:22:52-orch:cephadm-wip-guits-testing-4-2024-01-31-1839-distro-default-smithi/7541450/teuthology.log Found 8 distinct failure reasons Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi071 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=01c10bedc1108cdeb3a29239ed5b837f5b75f0b5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 1 jobs: ['7541483'] suites: ['agent/off', 'mon_election/classic', 'orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools', 'task/test_cephadm}'] Dead: None 2 jobs: ['7541450', '7541451'] Failure: reached maximum tries (301) after waiting for 300 seconds 1 jobs: ['7541478'] suites: ['0-nvme-loop', '1-start', '2-services/jaeger', '3-final}', 'orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools'] Failure: SSH connection to smithi203 was lost: 'sudo yum install -y kernel' 1 jobs: ['7541452'] suites: ['0-nvme-loop', '1-start', '2-ops/rmdir-reactivate}', 'orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8'] Failure: '2024-02-01T16:29:57.272238+00:00 smithi112 kernel: rcu: INFO: rcu_sched detected stalls on CPUs/tasks: ' in syslog 1 jobs: ['7541491'] suites: ['1-bootstrap/16.2.5', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools'] Failure: Failed to reconnect to smithi096 1 jobs: ['7541494'] suites: ['0-nvme-loop', '1-start', '2-services/nfs-ingress-rgw-user', '3-final}', 'orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04'] Failure: Command failed on smithi145 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:01c10bedc1108cdeb3a29239ed5b837f5b75f0b5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 181abeee-c119-11ee-95b6-87774f69a715 -- ceph orch ls -f json' 1 jobs: ['7541475'] suites: ['1-start', '2-services/rgw', '3-final}', 'orch:cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0}'] Failure: Test failure: test_daemon_restart (tasks.cephadm_cases.test_cli.TestCephadmCLI) 1 jobs: ['7541503'] suites: ['agent/on', 'mon_election/connectivity', 'orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8', 'task/test_orch_cli}']