2021-03-15T02:30:33.642 INFO:root:teuthology version: 1.1.0-b9656917 2021-03-15T02:30:33.646 INFO:teuthology.results:subset: '999/9999' 2021-03-15T02:30:33.646 INFO:teuthology.results:seed: '2477' 2021-03-15T03:07:42.068 INFO:root:teuthology version: 1.1.0-b9656917 2021-03-15T03:07:42.104 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-03-15T09:21:10.747 INFO:teuthology.results:Tests finished! gathering results... 2021-03-15T09:21:16.668 INFO:scrape:Found 50 jobs 2021-03-15T09:21:22.908 INFO:scrape:Found 7 distinct failure reasons 2021-03-15T09:21:22.909 INFO:scrape:Failure: Command failed (workunit test cephadm/create_iscsi_disks.sh) on smithi074 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86d577d82dbe15dd1793647ff36565039110c835 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/cephadm/create_iscsi_disks.sh' 2021-03-15T09:21:22.910 INFO:scrape:4 jobs: ['5965737', '5965721', '5965713', '5965748'] 2021-03-15T09:21:22.910 INFO:scrape:suites intersection: ['task/test_e2e}'] 2021-03-15T09:21:22.911 INFO:scrape:suites union: ['rados/cephadm/dashboard/{0-distro/ubuntu_20.04_kubic_stable', 'rados/cephadm/dashboard/{0-distro/ubuntu_20.04_kubic_testing', 'task/test_e2e}'] 2021-03-15T09:21:22.911 INFO:scrape: 2021-03-15T09:21:22.912 INFO:scrape:Failure: Test failure: test_cephfs_mirror (tasks.cephadm_cases.test_cli.TestCephadmCLI) 2021-03-15T09:21:22.912 INFO:scrape:3 jobs: ['5965723', '5965744', '5965711'] 2021-03-15T09:21:22.913 INFO:scrape:suites intersection: ['task/test_orch_cli}'] 2021-03-15T09:21:22.913 INFO:scrape:suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/ubuntu_20.04_kubic_stable', 'rados/cephadm/workunits/{0-distro/ubuntu_20.04_kubic_testing', 'task/test_orch_cli}'] 2021-03-15T09:21:22.913 INFO:scrape: 2021-03-15T09:21:22.914 INFO:scrape:Failure: Command failed (workunit test scrub/osd-scrub-repair.sh) on smithi066 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86d577d82dbe15dd1793647ff36565039110c835 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh' 2021-03-15T09:21:22.914 INFO:scrape:1 jobs: ['5965728'] 2021-03-15T09:21:22.915 INFO:scrape:suites: ['rados/standalone/{mon_election/classic', 'supported-random-distro$/{centos_8}', 'workloads/scrub}'] 2021-03-15T09:21:22.915 INFO:scrape: 2021-03-15T09:21:22.916 INFO:scrape:Failure: Command failed on smithi081 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:7cca4afa9cb86728910b57fe90aa38f637fdd5f6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 05e41362-853d-11eb-908f-001a4aab830c -- ceph orch host ls --format=json' 2021-03-15T09:21:22.916 INFO:scrape:2 jobs: ['5965746', '5965752'] 2021-03-15T09:21:22.916 INFO:scrape:suites intersection: ['1-start', '2-thrash', 'fixed-2', 'root}'] 2021-03-15T09:21:22.917 INFO:scrape:suites union: ['1-start', '2-thrash', '3-tasks/small-objects', '3-tasks/snaps-few-objects', 'fixed-2', 'msgr/async', 'msgr/async-v2only', 'rados/cephadm/thrash/{0-distro/ubuntu_20.04_kubic_stable', 'rados/cephadm/thrash/{0-distro/ubuntu_20.04_kubic_testing', 'root}'] 2021-03-15T09:21:22.917 INFO:scrape: 2021-03-15T09:21:22.918 INFO:scrape:Dead: b'2021-03-15T09:18:40.847 DEBUG:teuthology.exit:Finished running handlers' 2021-03-15T09:21:22.918 INFO:scrape:6 jobs: ['5965732', '5965722', '5965743', '5965715', '5965741', '5965730'] 2021-03-15T09:21:22.919 INFO:scrape:suites intersection: [] 2021-03-15T09:21:22.919 INFO:scrape:suites union: ['2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/classic}', 'mon_election/connectivity', 'mon_election/connectivity}', 'msgr-failures/few', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore-comp-lz4', 'objectstore/filestore-xfs', 'openstack}', 'rados', 'rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'tasks/rados_cls_all', 'thrashosds-health}', 'validater/valgrind}'] 2021-03-15T09:21:22.919 INFO:scrape: 2021-03-15T09:21:22.920 INFO:scrape:Crash: Command failed (workunit test rados/test_rados_tool.sh) on smithi040 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86d577d82dbe15dd1793647ff36565039110c835 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_rados_tool.sh' 2021-03-15T09:21:22.920 INFO:scrape:ceph version 17.0.0-2024-g7cca4afa (7cca4afa9cb86728910b57fe90aa38f637fdd5f6) quincy (dev) 1: /lib64/libpthread.so.0(+0x12b20) [0x7f1624c61b20] 2: gsignal() 3: abort() 4: /lib64/libstdc++.so.6(+0x9009b) [0x7f162427f09b] 5: /lib64/libstdc++.so.6(+0x9653c) [0x7f162428553c] 6: /lib64/libstdc++.so.6(+0x95559) [0x7f1624284559] 7: __gxx_personality_v0() 8: /lib64/libgcc_s.so.1(+0x10b13) [0x7f1623c65b13] 9: _Unwind_Resume() 10: ceph-osd(+0x564f0e) [0x55c3ba246f0e] 11: (ShardedThreadPool::WorkThreadSharded::entry()+0x14) [0x55c3ba9cdbf4] 12: (Thread::_entry_func(void*)+0xd) [0x55c3ba9bca5d] 13: /lib64/libpthread.so.0(+0x814a) [0x7f1624c5714a] 14: clone() 2021-03-15T09:21:22.921 INFO:scrape:1 jobs: ['5965747'] 2021-03-15T09:21:22.921 INFO:scrape:suites: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-comp-zstd', 'rados', 'rados/singleton/{all/radostool', 'supported-random-distro$/{rhel_8}}'] 2021-03-15T09:21:22.922 INFO:scrape: 2021-03-15T09:21:22.922 INFO:scrape:Failure: Command failed (workunit test rados/test.sh) on smithi051 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86d577d82dbe15dd1793647ff36565039110c835 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 ALLOW_TIMEOUTS=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 2021-03-15T09:21:22.922 INFO:scrape:1 jobs: ['5965749'] 2021-03-15T09:21:22.923 INFO:scrape:suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/default/{default', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-comp-zlib', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'thrashosds-health}', 'validater/valgrind}'] 2021-03-15T09:21:22.923 INFO:scrape: