2021-10-13T17:18:53.050 INFO:root:teuthology version: 1.1.0-e4202151 2021-10-13T17:18:53.050 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-10-14T04:27:44.864 INFO:teuthology.results:Tests finished! gathering results... 2021-10-14T04:27:51.796 INFO:scrape:Found 80 jobs 2021-10-14T04:27:53.274 INFO:scrape:Found 6 distinct failure reasons 2021-10-14T04:27:53.274 INFO:scrape:Failure: 'check osd count' reached maximum tries (90) after waiting for 900 seconds 2021-10-14T04:27:53.274 INFO:scrape:10 jobs: ['6440752', '6440754', '6440750', '6440747', '6440749', '6440746', '6440753', '6440745', '6440748', '6440751'] 2021-10-14T04:27:53.274 INFO:scrape:suites intersection: ['0-kubeadm', '1-rook', '2-workload/none', '3-final', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/master}'] 2021-10-14T04:27:53.275 INFO:scrape:suites union: ['0-kubeadm', '1-rook', '2-workload/none', '3-final', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/master}'] 2021-10-14T04:27:53.275 INFO:scrape: 2021-10-14T04:27:53.275 INFO:scrape:Dead: b'2021-10-14T04:24:41.532 DEBUG:teuthology.exit:Finished running handlers' 2021-10-14T04:27:53.275 INFO:scrape:1 jobs: ['6440697'] 2021-10-14T04:27:53.275 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'thrashosds-health}', 'validater/valgrind}'] 2021-10-14T04:27:53.275 INFO:scrape: 2021-10-14T04:27:53.275 INFO:scrape:Failure: Command failed (workunit test rados/test.sh) on smithi191 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=19a35d5b2458f2f5e3a349c4eadfef37c2b7630d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 2021-10-14T04:27:53.276 INFO:scrape:3 jobs: ['6440704', '6440701', '6440696'] 2021-10-14T04:27:53.276 INFO:scrape:suites intersection: ['ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'thrashosds-health}', 'validater/valgrind}'] 2021-10-14T04:27:53.276 INFO:scrape:suites union: ['ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'thrashosds-health}', 'validater/valgrind}'] 2021-10-14T04:27:53.276 INFO:scrape: 2021-10-14T04:27:53.276 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh 2021-10-14T04:27:53.276 INFO:scrape:1 jobs: ['6440726'] 2021-10-14T04:27:53.276 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_cls_all', 'thrashosds-health}', 'validater/valgrind}'] 2021-10-14T04:27:53.277 INFO:scrape: 2021-10-14T04:27:53.277 INFO:scrape:Failure: Command failed on smithi064 with status 127: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e72021f8-2c46-11ec-8c27-001a4aab830c -- ceph orch apply mon '2;smithi188:172.21.15.188=smithi188;smithi064:172.21.15.64=smithi064'" 2021-10-14T04:27:53.277 INFO:scrape:1 jobs: ['6440707'] 2021-10-14T04:27:53.277 INFO:scrape:suites: ['1-bootstrap/16.2.5', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0'] 2021-10-14T04:27:53.277 INFO:scrape: 2021-10-14T04:27:53.277 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2021-10-14T04:27:53.277 INFO:scrape:1 jobs: ['6440727'] 2021-10-14T04:27:53.277 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_cls_all', 'thrashosds-health}', 'validater/valgrind}'] 2021-10-14T04:27:53.278 INFO:scrape: