2021-03-03T01:55:44.441 INFO:root:teuthology version: 1.1.0-3082387b
2021-03-03T01:55:44.443 INFO:teuthology.results:subset: '1/50000'
2021-03-03T01:55:44.443 INFO:teuthology.results:seed: '2364'
2021-03-03T02:28:36.323 INFO:root:teuthology version: 1.1.0-3082387b
2021-03-03T02:28:36.355 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-03-03T14:27:30.847 INFO:teuthology.results:Tests finished! gathering results...
2021-03-03T14:28:32.895 INFO:scrape:Found 23 jobs
2021-03-03T14:29:27.638 INFO:scrape:Found 9 distinct failure reasons
2021-03-03T14:29:27.640 INFO:scrape:Failure: Command failed (workunit test cephtool/test.sh) on smithi039 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5c9fbc794811a1de66ee254a1f9078372839e4f3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'
2021-03-03T14:29:27.640 INFO:scrape:1 jobs: ['5927775']
2021-03-03T14:29:27.641 INFO:scrape:suites: ['mon_election/classic', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'rados', 'rados/singleton-bluestore/{all/cephtool', 'supported-random-distro$/{centos_8}}']
2021-03-03T14:29:27.641 INFO:scrape:
2021-03-03T14:29:27.642 INFO:scrape:Dead: b'2021-03-03T14:02:15.391 DEBUG:teuthology.exit:Finished running handlers'
2021-03-03T14:29:27.642 INFO:scrape:8 jobs: ['5927762', '5927777', '5927773', '5927768', '5927774', '5927782', '5927764', '5927767']
2021-03-03T14:29:27.643 INFO:scrape:suites intersection: ['msgr/async-v2only', 'rados']
2021-03-03T14:29:27.643 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', '2-recovery-overrides/{more-active-recovery}', '2-recovery-overrides/{more-async-partial-recovery}', 'backoff/peering', 'ceph', 'clusters/9-mons', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/crush-compat', 'd-balancer/on', 'd-thrash/default/{default', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr-failures/many', 'msgr-failures/mon-delay', 'msgr/async-v2only', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/monthrash/{ceph', 'rados/singleton/{all/divergent_priors', 'rados/singleton/{all/lost-unfound', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'rados/verify/{centos_latest', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{rhel_8}}', 'tasks/mon_recovery', 'thrashers/default', 'thrashers/none', 'thrashers/sync', 'thrashosds-health', 'thrashosds-health}', 'validater/lockdep}', 'workloads/pool-create-delete}', 'workloads/rados_api_tests}', 'workloads/redirect}', 'workloads/small-objects}']
2021-03-03T14:29:27.643 INFO:scrape:
2021-03-03T14:29:27.644 INFO:scrape:Failure: 
2021-03-03T14:29:27.644 INFO:scrape:1 jobs: ['5927784']
2021-03-03T14:29:27.645 INFO:scrape:suites: ['mon_election/classic', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-stupid', 'rados', 'rados/singleton/{all/admin-socket', 'supported-random-distro$/{rhel_8}}']
2021-03-03T14:29:27.645 INFO:scrape:
2021-03-03T14:29:27.646 INFO:scrape:Failure: "2021-03-03T02:16:31.099149+0000 mon.a (mon.0) 174 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log
2021-03-03T14:29:27.646 INFO:scrape:2 jobs: ['5927769', '5927763']
2021-03-03T14:29:27.647 INFO:scrape:suites intersection: ['msgr/async-v2only', 'rados', 'rados/monthrash/{ceph']
2021-03-03T14:29:27.647 INFO:scrape:suites union: ['clusters/3-mons', 'clusters/9-mons', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr-failures/mon-delay', 'msgr/async-v2only', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-zstd', 'rados', 'rados/monthrash/{ceph', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'thrashers/force-sync-many', 'thrashers/sync-many', 'workloads/rados_5925}', 'workloads/rados_mon_workunits}']
2021-03-03T14:29:27.647 INFO:scrape:
2021-03-03T14:29:27.648 INFO:scrape:Failure: Command failed on smithi016 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log'
2021-03-03T14:29:27.648 INFO:scrape:1 jobs: ['5927779']
2021-03-03T14:29:27.649 INFO:scrape:suites: ['mon_election/classic', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-lz4', 'rados', 'rados/singleton/{all/test_envlibrados_for_rocksdb', 'supported-random-distro$/{rhel_8}}']
2021-03-03T14:29:27.649 INFO:scrape:
2021-03-03T14:29:27.649 INFO:scrape:Failure: Command failed (workunit test mon/pg_autoscaler.sh) on smithi131 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5c9fbc794811a1de66ee254a1f9078372839e4f3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mon/pg_autoscaler.sh'
2021-03-03T14:29:27.650 INFO:scrape:1 jobs: ['5927776']
2021-03-03T14:29:27.650 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/many', 'msgr/async-v2only', 'objectstore/bluestore-comp-lz4', 'rados', 'rados/singleton/{all/pg-autoscaler-progress-off', 'supported-random-distro$/{centos_8}}']
2021-03-03T14:29:27.651 INFO:scrape:
2021-03-03T14:29:27.651 INFO:scrape:Failure: timeout expired in wait_until_healthy
2021-03-03T14:29:27.652 INFO:scrape:3 jobs: ['5927766', '5927781', '5927772']
2021-03-03T14:29:27.652 INFO:scrape:suites intersection: ['fixed-2', 'msgr/async-v2only', 'start']
2021-03-03T14:29:27.653 INFO:scrape:suites union: ['fixed-2', 'mode/packaged', 'mode/root', 'mon_election/classic', 'mon_election/connectivity', 'msgr/async-v2only', 'rados/cephadm/with-work/{distro/centos_latest', 'rados/cephadm/with-work/{distro/rhel_8.0', 'rados/cephadm/with-work/{distro/rhel_latest', 'start', 'tasks/rados_api_tests}', 'tasks/rados_python}']
2021-03-03T14:29:27.653 INFO:scrape:
2021-03-03T14:29:27.653 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rados/test_python.sh
2021-03-03T14:29:27.654 INFO:scrape:1 jobs: ['5927765']
2021-03-03T14:29:27.654 INFO:scrape:suites: ['clusters/{fixed-2', 'mon_election/connectivity', 'msgr-failures/many', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rados', 'rados/basic/{ceph', 'supported-random-distro$/{rhel_8}', 'tasks/rados_python}']
2021-03-03T14:29:27.655 INFO:scrape:
2021-03-03T14:29:27.655 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rados/test.sh
2021-03-03T14:29:27.656 INFO:scrape:1 jobs: ['5927780']
2021-03-03T14:29:27.656 INFO:scrape:suites: ['clusters/{fixed-2', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rados', 'rados/basic/{ceph', 'supported-random-distro$/{rhel_8}', 'tasks/rados_api_tests}']
2021-03-03T14:29:27.657 INFO:scrape: