2021-12-16T08:28:03.251 INFO:root:teuthology version: 1.1.0-95a7d479
2021-12-16T08:28:03.253 INFO:teuthology.results:subset: '1/80000'
2021-12-16T08:28:03.253 INFO:teuthology.results:seed: '1203'
2021-12-16T16:17:16.898 INFO:root:teuthology version: 1.1.0-95a7d479
2021-12-16T16:17:16.932 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-12-17T03:41:18.303 INFO:teuthology.results:Tests finished! gathering results...
2021-12-17T03:41:56.315 INFO:scrape:Found 404 jobs
2021-12-17T03:42:00.201 INFO:scrape:Found 10 distinct failure reasons
2021-12-17T03:42:00.202 INFO:scrape:Crash: Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_msgr'
2021-12-17T03:42:00.203 INFO:scrape:ceph version 17.0.0-9642-g5918bc24 (5918bc24950acdd68e1119c7b50cf19bfc4aeab9) quincy (dev)
 1: /lib64/libpthread.so.0(+0x12c20) [0x7fedc4e3bc20]
 2: gsignal()
 3: abort()
 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x7fedc3420d12]
 5: /usr/lib64/ceph/libceph-common.so.2(+0x281ed5) [0x7fedc3420ed5]
 6: /usr/lib64/ceph/libceph-common.so.2(+0x5bbf9d) [0x7fedc375af9d]
 7: (AsyncMessenger::~AsyncMessenger()+0xd) [0x7fedc375afad]
 8: (MessengerTest::TearDown()+0xdc) [0x55c8aa08183c]
 9: (void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x4e) [0x55c8aa0cf8fe]
 10: (testing::TestInfo::Run()+0x135) [0x55c8aa0c2695]
 11: (testing::TestSuite::Run()+0xc1) [0x55c8aa0c2bf1]
 12: (testing::internal::UnitTestImpl::RunAllTests()+0x445) [0x55c8aa0c4805]
 13: (bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*)+0x4e) [0x55c8aa0cfe6e]
 14: (testing::UnitTest::Run()+0xa0) [0x55c8aa0c2760]
 15: main()
 16: __libc_start_main()
 17: _start()
2021-12-17T03:42:00.203 INFO:scrape:1 jobs: ['6565681']
2021-12-17T03:42:00.204 INFO:scrape:suites: ['mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/msgr', 'supported-random-distro$/{centos_8.stream}}']
2021-12-17T03:42:00.204 INFO:scrape:
2021-12-17T03:42:00.205 INFO:scrape:Failure: No module named 'tasks.ceph'
2021-12-17T03:42:00.206 INFO:scrape:5 jobs: ['6565382', '6565402', '6565385', '6565404', '6565376']
2021-12-17T03:42:00.206 INFO:scrape:suites intersection: []
2021-12-17T03:42:00.207 INFO:scrape:suites union: ['ceph', 'clusters/{fixed-2', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr-failures/many', 'msgr-failures/none', 'msgr-failures/osd-dispatch-delay', 'msgr/async', 'msgr/async-v1only', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/filestore-xfs', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/cephadm/workunits/{agent/on', 'rados/singleton/{all/dump-stuck', 'rados/singleton/{all/ec-lost-unfound', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'recovery-overrides/{more-async-recovery}', 'supported-random-distro$/{centos_8.stream}}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_latest}', 'supported-random-distro$/{ubuntu_latest}}', 'task/test_orch_cli}', 'tasks/rados_python}', 'thrashers/pggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=isa-k=2-m=1}']
2021-12-17T03:42:00.207 INFO:scrape:
2021-12-17T03:42:00.208 INFO:scrape:Failure: Command failed on smithi063 with status 126: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:5918bc24950acdd68e1119c7b50cf19bfc4aeab9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 93e0f24e-5e71-11ec-8c31-001a4aab830c -- ceph mon dump -f json'
2021-12-17T03:42:00.208 INFO:scrape:2 jobs: ['6565515', '6565484']
2021-12-17T03:42:00.209 INFO:scrape:suites intersection: ['0-nvme-loop', '1-start']
2021-12-17T03:42:00.209 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-ops/rm-zap-flag}', '2-services/client-keyring', '3-final}', 'rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8', 'rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools']
2021-12-17T03:42:00.210 INFO:scrape:
2021-12-17T03:42:00.211 INFO:scrape:Failure: Command failed on smithi070 with status 1: 'sudo yum -y install cephfs-java'
2021-12-17T03:42:00.211 INFO:scrape:1 jobs: ['6565387']
2021-12-17T03:42:00.212 INFO:scrape:suites: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-active-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-3}', 'backoff/peering', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/on', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'msgr/async-v2only', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_8}', 'thrashers/default', 'thrashosds-health', 'workloads/small-objects-balanced}']
2021-12-17T03:42:00.212 INFO:scrape:
2021-12-17T03:42:00.213 INFO:scrape:Dead: b'2021-12-17T00:46:26.338 DEBUG:teuthology.exit:Finished running handlers'
2021-12-17T03:42:00.213 INFO:scrape:2 jobs: ['6565490', '6565637']
2021-12-17T03:42:00.214 INFO:scrape:suites intersection: []
2021-12-17T03:42:00.214 INFO:scrape:suites union: ['1-bootstrap/octopus', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools', 'rados/objectstore/{backends/objectstore-bluestore-a', 'supported-random-distro$/{centos_8.stream}}']
2021-12-17T03:42:00.215 INFO:scrape:
2021-12-17T03:42:00.215 INFO:scrape:Failure: [Errno 2] Cannot find file on the remote 'ubuntu@smithi059.front.sepia.ceph.com': 'rook/cluster/examples/kubernetes/ceph/operator.yaml'
2021-12-17T03:42:00.216 INFO:scrape:2 jobs: ['6565441', '6565644']
2021-12-17T03:42:00.216 INFO:scrape:suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/radosbench', '3-final', 'k8s/1.21', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/master}']
2021-12-17T03:42:00.217 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/radosbench', '3-final', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/flannel', 'net/host', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/master}']
2021-12-17T03:42:00.217 INFO:scrape:
2021-12-17T03:42:00.217 INFO:scrape:Failure: Test failure: test_module_commands (tasks.mgr.test_module_selftest.TestModuleSelftest)
2021-12-17T03:42:00.218 INFO:scrape:1 jobs: ['6565430']
2021-12-17T03:42:00.218 INFO:scrape:suites: ['debug/mgr', 'mon_election/connectivity', 'rados/mgr/{clusters/{2-node-mgr}', 'random-objectstore$/{bluestore-comp-lz4}', 'supported-random-distro$/{centos_8}', 'tasks/module_selftest}']
2021-12-17T03:42:00.219 INFO:scrape:
2021-12-17T03:42:00.219 INFO:scrape:Failure: expected MON_CLOCK_SKEW but got none
2021-12-17T03:42:00.220 INFO:scrape:1 jobs: ['6565579']
2021-12-17T03:42:00.220 INFO:scrape:suites: ['mon_election/classic', 'msgr-failures/few', 'msgr/async-v2only', 'no_pools', 'objectstore/bluestore-stupid', 'rados', 'rados/multimon/{clusters/21', 'supported-random-distro$/{centos_8.stream}', 'tasks/mon_clock_with_skews}']
2021-12-17T03:42:00.221 INFO:scrape:
2021-12-17T03:42:00.221 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi031 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5918bc24950acdd68e1119c7b50cf19bfc4aeab9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh'
2021-12-17T03:42:00.222 INFO:scrape:2 jobs: ['6565645', '6565442']
2021-12-17T03:42:00.222 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-stupid}', 'tasks/e2e}']
2021-12-17T03:42:00.223 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-stupid}', 'tasks/e2e}']
2021-12-17T03:42:00.223 INFO:scrape:
2021-12-17T03:42:00.223 INFO:scrape:Failure: Command failed on smithi187 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.0.0-9642-g5918bc24-1focal ceph-mds=17.0.0-9642-g5918bc24-1focal ceph-common=17.0.0-9642-g5918bc24-1focal ceph-fuse=17.0.0-9642-g5918bc24-1focal ceph-test=17.0.0-9642-g5918bc24-1focal radosgw=17.0.0-9642-g5918bc24-1focal python-ceph=17.0.0-9642-g5918bc24-1focal libcephfs1=17.0.0-9642-g5918bc24-1focal libcephfs-java=17.0.0-9642-g5918bc24-1focal libcephfs-jni=17.0.0-9642-g5918bc24-1focal librados2=17.0.0-9642-g5918bc24-1focal librbd1=17.0.0-9642-g5918bc24-1focal rbd-fuse=17.0.0-9642-g5918bc24-1focal'
2021-12-17T03:42:00.224 INFO:scrape:1 jobs: ['6565388']
2021-12-17T03:42:00.224 INFO:scrape:suites: ['mon_election/classic', 'objectstore/bluestore-bitmap', 'openstack', 'rados/perf/{ceph', 'scheduler/dmclock_default_shards', 'settings/optimized', 'ubuntu_latest', 'workloads/fio_4M_rand_rw}']
2021-12-17T03:42:00.225 INFO:scrape: