2021-12-20T21:36:28.380 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-20T21:36:28.383 INFO:teuthology.results:subset: '1/80000' 2021-12-20T21:36:28.383 INFO:teuthology.results:seed: '1203' 2021-12-21T01:27:54.129 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-21T01:27:54.157 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-12-21T13:13:01.555 INFO:teuthology.results:Tests finished! gathering results... 2021-12-21T13:13:37.421 INFO:scrape:Found 385 jobs 2021-12-21T13:13:41.377 INFO:scrape:Found 6 distinct failure reasons 2021-12-21T13:13:41.378 INFO:scrape:Failure: Command failed (workunit test cephtool/test.sh) on smithi185 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2d565dd83247ceee9ec28986c8a3b1639d291f98 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh' 2021-12-21T13:13:41.379 INFO:scrape:4 jobs: ['6573717', '6573804', '6573994', '6573900'] 2021-12-21T13:13:41.379 INFO:scrape:suites intersection: ['rados', 'rados/singleton-bluestore/{all/cephtool'] 2021-12-21T13:13:41.380 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr-failures/many', 'msgr-failures/none', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'rados', 'rados/singleton-bluestore/{all/cephtool', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{ubuntu_latest}}'] 2021-12-21T13:13:41.380 INFO:scrape: 2021-12-21T13:13:41.381 INFO:scrape:Dead: b'2021-12-21T11:07:16.312 DEBUG:teuthology.exit:Finished running handlers' 2021-12-21T13:13:41.381 INFO:scrape:2 jobs: ['6573776', '6573972'] 2021-12-21T13:13:41.382 INFO:scrape:suites intersection: ['1-bootstrap/octopus', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools'] 2021-12-21T13:13:41.382 INFO:scrape:suites union: ['1-bootstrap/octopus', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools'] 2021-12-21T13:13:41.383 INFO:scrape: 2021-12-21T13:13:41.383 INFO:scrape:Failure: timeout expired in wait_until_healthy 2021-12-21T13:13:41.383 INFO:scrape:2 jobs: ['6573928', '6573779'] 2021-12-21T13:13:41.384 INFO:scrape:suites intersection: ['rados/cephadm/workunits/{agent/on', 'task/test_nfs}'] 2021-12-21T13:13:41.384 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{agent/on', 'task/test_nfs}'] 2021-12-21T13:13:41.385 INFO:scrape: 2021-12-21T13:13:41.385 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi125 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2d565dd83247ceee9ec28986c8a3b1639d291f98 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2021-12-21T13:13:41.386 INFO:scrape:2 jobs: ['6573734', '6573926'] 2021-12-21T13:13:41.386 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-stupid}', 'tasks/e2e}'] 2021-12-21T13:13:41.387 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-stupid}', 'tasks/e2e}'] 2021-12-21T13:13:41.387 INFO:scrape: 2021-12-21T13:13:41.388 INFO:scrape:Failure: Command failed (workunit test mon/crush_ops.sh) on smithi078 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2d565dd83247ceee9ec28986c8a3b1639d291f98 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/crush_ops.sh' 2021-12-21T13:13:41.388 INFO:scrape:1 jobs: ['6573867'] 2021-12-21T13:13:41.388 INFO:scrape:suites: ['clusters/3-mons', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-stupid', 'rados', 'rados/monthrash/{ceph', 'supported-random-distro$/{centos_8.stream}', 'thrashers/many', 'workloads/rados_mon_workunits}'] 2021-12-21T13:13:41.389 INFO:scrape: 2021-12-21T13:13:41.389 INFO:scrape:Failure: reached maximum tries (800) after waiting for 4800 seconds 2021-12-21T13:13:41.390 INFO:scrape:1 jobs: ['6573985'] 2021-12-21T13:13:41.390 INFO:scrape:suites: ['clusters/{fixed-2', 'fast/normal', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'objectstore/bluestore-comp-lz4', 'openstack}', 'rados', 'rados/thrash-erasure-code/{ceph', 'recovery-overrides/{default}', 'supported-random-distro$/{centos_8.stream}', 'thrashers/default', 'thrashosds-health', 'workloads/ec-radosbench}'] 2021-12-21T13:13:41.391 INFO:scrape: