2022-01-24T22:42:49.622 INFO:root:teuthology version: 1.1.0-7c0cb867 2022-01-24T22:42:49.623 INFO:teuthology.results:subset: '111/120000' 2022-01-24T22:42:49.624 INFO:teuthology.results:seed: '6552' 2022-01-24T22:54:20.041 INFO:root:teuthology version: 1.1.0-7c0cb867 2022-01-24T22:54:20.052 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-01-25T05:34:50.442 INFO:teuthology.results:Tests finished! gathering results... 2022-01-25T05:34:52.627 INFO:scrape:Found 26 jobs 2022-01-25T05:34:54.080 INFO:scrape:Found 5 distinct failure reasons 2022-01-25T05:34:54.081 INFO:scrape:Dead: b'2022-01-25T05:22:53.821 DEBUG:teuthology.exit:Finished running handlers' 2022-01-25T05:34:54.081 INFO:scrape:8 jobs: ['6638252', '6638261', '6638248', '6638251', '6638268', '6638257', '6638245', '6638264'] 2022-01-25T05:34:54.082 INFO:scrape:suites intersection: ['1-start', '2-nfs', '3-upgrade-with-workload', '4-final}'] 2022-01-25T05:34:54.082 INFO:scrape:suites union: ['1-bootstrap/16.2.4', '1-bootstrap/octopus', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.3_container_tools_3.0', 'rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools'] 2022-01-25T05:34:54.083 INFO:scrape: 2022-01-25T05:34:54.083 INFO:scrape:Failure: SSH connection to smithi033 was lost: 'uname -r' 2022-01-25T05:34:54.083 INFO:scrape:4 jobs: ['6638260', '6638249', '6638254', '6638263'] 2022-01-25T05:34:54.084 INFO:scrape:suites intersection: [] 2022-01-25T05:34:54.084 INFO:scrape:suites union: ['0-nvme-loop', '1-install/luminous', '1-start', '2-ops/rm-zap-add}', '2-services/nfs-ingress2', '3-final}', 'backoff/peering', 'ceph', 'clusters/{openstack', 'd-balancer/on', 'distro$/{ubuntu_18.04}', 'distro/ubuntu_18.04', 'fixed-2', 'mon_election/connectivity', 'msgr-failures/few', 'rados', 'rados/cephadm/osds/{0-distro/ubuntu_18.04', 'rados/cephadm/smoke-roleless/{0-distro/ubuntu_18.04', 'rados/cephadm/smoke/{0-nvme-loop', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'start}', 'thrashers/morepggrow', 'thrashosds-health', 'three-plus-one}', 'workloads/cache-snaps}'] 2022-01-25T05:34:54.085 INFO:scrape: 2022-01-25T05:34:54.085 INFO:scrape:Failure: [Errno 2] Cannot find file on the remote 'ubuntu@smithi036.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml' 2022-01-25T05:34:54.086 INFO:scrape:1 jobs: ['6638258'] 2022-01-25T05:34:54.086 INFO:scrape:suites: ['0-kubeadm', '1-rook', '2-workload/none', '3-final', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/master}'] 2022-01-25T05:34:54.087 INFO:scrape: 2022-01-25T05:34:54.087 INFO:scrape:Failure: Command failed on smithi100 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f79bf06-7d69-11ec-8c35-001a4aab830c -- bash -c \'ceph --format=json mds versions | jq -e ". | add == 4"\'' 2022-01-25T05:34:54.088 INFO:scrape:1 jobs: ['6638244'] 2022-01-25T05:34:54.088 INFO:scrape:suites: ['1-ranks/1', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'roles', 'tasks/{0-from/v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-01-25T05:34:54.089 INFO:scrape: 2022-01-25T05:34:54.089 INFO:scrape:Failure: Command failed on smithi135 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:656fd1b72f2e6d6132afd224e66dc1ee26b8d0c8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0d01ea86-7d6a-11ec-8c35-001a4aab830c -- bash -c \'set -e\nset -x\nceph orch ps\nceph orch device ls\nDEVID=$(ceph device ls | grep osd.1 | awk \'"\'"\'{print $1}\'"\'"\')\nHOST=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $1}\'"\'"\')\nDEV=$(ceph orch device ls | grep $DEVID | awk \'"\'"\'{print $2}\'"\'"\')\necho "host $HOST, dev $DEV, devid $DEVID"\nceph orch osd rm 1\nwhile ceph orch osd rm status | grep ^1 ; do sleep 5 ; done\nceph orch device zap $HOST $DEV --force\nwhile ! ceph osd dump | grep osd.1 | grep up ; do sleep 5 ; done\n\'' 2022-01-25T05:34:54.089 INFO:scrape:1 jobs: ['6638266'] 2022-01-25T05:34:54.090 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/rm-zap-wait}', 'rados/cephadm/osds/{0-distro/ubuntu_18.04'] 2022-01-25T05:34:54.090 INFO:scrape: