2021-09-02T07:55:11.882 INFO:root:teuthology version: 1.1.0-df69d6db 2021-09-02T07:55:11.883 INFO:teuthology.results:subset: '111/120000' 2021-09-02T07:55:11.884 INFO:teuthology.results:seed: '254' 2021-09-02T08:08:38.268 INFO:root:teuthology version: 1.1.0-df69d6db 2021-09-02T08:08:38.301 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-09-02T20:09:28.590 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2021-09-02T20:09:30.720 INFO:scrape:Found 25 jobs 2021-09-02T20:09:32.252 INFO:scrape:Found 5 distinct failure reasons 2021-09-02T20:09:32.253 INFO:scrape:Failure: Command failed (workunit test cls/test_cls_rgw.sh) on smithi035 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=pacific TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh' 2021-09-02T20:09:32.254 INFO:scrape:2 jobs: ['6372398', '6372399'] 2021-09-02T20:09:32.254 INFO:scrape:suites intersection: ['mon_election/connectivity'] 2021-09-02T20:09:32.254 INFO:scrape:suites union: ['0-start', '1-tasks', 'ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rados', 'rados/upgrade/parallel/{0-distro$/{ubuntu_20.04}', 'rados/verify/{centos_latest', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'tasks/rados_cls_all', 'test_rbd_api', 'test_rbd_python}}', 'thrashosds-health}', 'upgrade-sequence', 'validater/valgrind}', 'workload/{ec-rados-default'] 2021-09-02T20:09:32.255 INFO:scrape: 2021-09-02T20:09:32.255 INFO:scrape:Failure: {'smithi123.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} 2021-09-02T20:09:32.256 INFO:scrape:4 jobs: ['6372377', '6372392', '6372395', '6372391'] 2021-09-02T20:09:32.256 INFO:scrape:suites intersection: ['objectstore/filestore-xfs'] 2021-09-02T20:09:32.257 INFO:scrape:suites union: ['clusters/{fixed-2', 'debug/mgr', 'fast/normal', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr-failures/none', 'msgr/async-v2only', 'objectstore/filestore-xfs', 'openstack}', 'rados', 'rados/mgr/{clusters/{2-node-mgr}', 'rados/singleton/{all/mon-config-keys', 'rados/singleton/{all/thrash-rados/{thrash-rados', 'rados/thrash-erasure-code/{ceph', 'recovery-overrides/{more-active-recovery}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{rhel_8}}', 'tasks/crash}', 'thrashers/fastread', 'thrashosds-health', 'thrashosds-health}', 'workloads/ec-radosbench}'] 2021-09-02T20:09:32.257 INFO:scrape: 2021-09-02T20:09:32.258 INFO:scrape:Failure: timeout expired in wait_for_all_osds_up 2021-09-02T20:09:32.258 INFO:scrape:1 jobs: ['6372390'] 2021-09-02T20:09:32.258 INFO:scrape:suites: ['0-kubeadm', '1-rook', '2-workload/none', '3-final', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/master}'] 2021-09-02T20:09:32.259 INFO:scrape: 2021-09-02T20:09:32.259 INFO:scrape:Failure: {'smithi134.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi180.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi036.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} 2021-09-02T20:09:32.260 INFO:scrape:2 jobs: ['6372380', '6372385'] 2021-09-02T20:09:32.260 INFO:scrape:suites intersection: ['objectstore/filestore-xfs', 'rados', 'supported-random-distro$/{rhel_8}'] 2021-09-02T20:09:32.261 INFO:scrape:suites union: ['clusters/{fixed-4', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async-v2only', 'no_pools', 'objectstore/filestore-xfs', 'openstack}', 'rados', 'rados/multimon/{clusters/9', 'rados/thrash-erasure-code-shec/{ceph', 'recovery-overrides/{more-active-recovery}', 'supported-random-distro$/{rhel_8}', 'tasks/mon_recovery}', 'thrashers/default', 'thrashosds-health', 'workloads/ec-rados-plugin=shec-k=4-m=3-c=2}'] 2021-09-02T20:09:32.261 INFO:scrape: 2021-09-02T20:09:32.262 INFO:scrape:Dead: b'2021-09-02T12:25:03.859 INFO:tasks.rook.operator.smithi036.stdout:2021-09-02 12:25:03.883596 W | op-mon: monitor e is not in quorum list' 2021-09-02T20:09:32.262 INFO:scrape:1 jobs: ['6372397'] 2021-09-02T20:09:32.263 INFO:scrape:suites: ['0-kubeadm', '1-rook', '2-workload/radosbench', '3-final', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.7.0}'] 2021-09-02T20:09:32.263 INFO:scrape: