2020-11-05T23:20:42.178 INFO:root:teuthology version: 1.0.0-41b569f 2020-11-05T23:20:42.179 INFO:teuthology.results:subset: '273/9999' 2020-11-05T23:20:42.180 INFO:teuthology.results:seed: '5098' 2020-11-05T23:23:10.815 INFO:root:teuthology version: 1.0.0-41b569f 2020-11-05T23:23:10.820 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-11-06T11:23:23.458 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2020-11-06T11:23:24.498 INFO:scrape:Found 7 jobs 2020-11-06T11:23:24.990 INFO:scrape:Found 2 distinct failure reasons 2020-11-06T11:23:24.991 INFO:scrape:Failure: 'CephManager' object has no attribute 'cephadm' 2020-11-06T11:23:24.991 INFO:scrape:4 jobs: ['5593947', '5593948', '5593951', '5593953'] 2020-11-06T11:23:24.991 INFO:scrape:suites intersection: ['ceph', 'clusters/{fixed-2', 'openstack}', 'rados', 'thrashosds-health'] 2020-11-06T11:23:24.992 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', '2-recovery-overrides/{more-active-recovery}', 'backoff/normal', 'backoff/peering', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/crush-compat', 'd-balancer/upmap', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async-v1only', 'msgr/random', 'msgr/simple', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados:thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados:thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{rhel_latest}', 'supported-random-distro$/{ubuntu_16.04}', 'thrashers/default', 'thrashers/mapgap', 'thrashers/pggrow', 'thrashosds-health', 'workloads/radosbench-high-concurrency}', 'workloads/radosbench}', 'workloads/redirect_set_object}', 'workloads/small-objects}'] 2020-11-06T11:23:24.992 INFO:scrape: 2020-11-06T11:23:24.992 INFO:scrape:Dead: b"teuthology.exceptions.ConnectionLostError: SSH connection to smithi066 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --set_chunk --low_tier_pool low_tier --max-ops 4000 --objects 300 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 10 --op write_excl 50 --pool unique_pool_0'" 2020-11-06T11:23:24.993 INFO:scrape:1 jobs: ['5593952'] 2020-11-06T11:23:24.993 INFO:scrape:suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/off', 'msgr-failures/fastclose', 'msgr/simple', 'objectstore/bluestore-comp-snappy', 'openstack}', 'rados', 'rados:thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{rhel_latest}', 'thrashers/careful', 'thrashosds-health', 'workloads/set-chunks-read}'] 2020-11-06T11:23:24.993 INFO:scrape: 2020-11-06T11:23:24.994 INFO:teuthology.results:Sending results to dupadhya@redhat.com: Test Run: ideepika-2020-11-05_23:20:11-rados:thrash-wip-yuri7-testing-2020-11-05-1949-nautilus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/ideepika-2020-11-05_23:20:11-rados:thrash-wip-yuri7-testing-2020-11-05-1949-nautilus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/ideepika-2020-11-05_23:20:11-rados:thrash-wip-yuri7-testing-2020-11-05-1949-nautilus-distro-basic-smithi/ failed: 4 dead: 0 running: 1 waiting: 0 queued: 0 passed: 2 Fail ================================================================= [5593947] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/upmap msgr-failures/few msgr/simple objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_latest} thrashers/default thrashosds-health workloads/radosbench-high-concurrency} ----------------------------------------------------------------- time: 00:57:12 info: http://pulpito.ceph.com/ideepika-2020-11-05_23:20:11-rados:thrash-wip-yuri7-testing-2020-11-05-1949-nautilus-distro-basic-smithi/5593947/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-11-05_23:20:11-rados:thrash-wip-yuri7-testing-2020-11-05-1949-nautilus-distro-basic-smithi/5593947/ 'CephManager' object has no attribute 'cephadm' [5593948] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_latest} thrashers/mapgap thrashosds-health workloads/radosbench} ----------------------------------------------------------------- time: 01:03:12 info: http://pulpito.ceph.com/ideepika-2020-11-05_23:20:11-rados:thrash-wip-yuri7-testing-2020-11-05-1949-nautilus-distro-basic-smithi/5593948/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-11-05_23:20:11-rados:thrash-wip-yuri7-testing-2020-11-05-1949-nautilus-distro-basic-smithi/5593948/ 'CephManager' object has no attribute 'cephadm' [5593951] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat msgr-failures/osd-delay msgr/random objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_latest} thrashers/pggrow thrashosds-health workloads/redirect_set_object} ----------------------------------------------------------------- time: 00:28:14 info: http://pulpito.ceph.com/ideepika-2020-11-05_23:20:11-rados:thrash-wip-yuri7-testing-2020-11-05-1949-nautilus-distro-basic-smithi/5593951/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-11-05_23:20:11-rados:thrash-wip-yuri7-testing-2020-11-05-1949-nautilus-distro-basic-smithi/5593951/ 'CephManager' object has no attribute 'cephadm' [5593953] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/upmap msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_16.04} thrashers/default thrashosds-health workloads/small-objects} ----------------------------------------------------------------- time: 00:32:31 info: http://pulpito.ceph.com/ideepika-2020-11-05_23:20:11-rados:thrash-wip-yuri7-testing-2020-11-05-1949-nautilus-distro-basic-smithi/5593953/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-11-05_23:20:11-rados:thrash-wip-yuri7-testing-2020-11-05-1949-nautilus-distro-basic-smithi/5593953/ 'CephManager' object has no attribute 'cephadm' Running ================================================================= [5593952] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/off msgr-failures/fastclose msgr/simple objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/set-chunks-read} info: http://pulpito.ceph.com/ideepika-2020-11-05_23:20:11-rados:thrash-wip-yuri7-testing-2020-11-05-1949-nautilus-distro-basic-smithi/5593952/ Pass ================================================================= [5593949] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/off msgr-failures/fastclose msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/redirect} time: 00:14:27 info: http://pulpito.ceph.com/ideepika-2020-11-05_23:20:11-rados:thrash-wip-yuri7-testing-2020-11-05-1949-nautilus-distro-basic-smithi/5593949/ [5593950] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect_promote_tests} time: 00:09:22 info: http://pulpito.ceph.com/ideepika-2020-11-05_23:20:11-rados:thrash-wip-yuri7-testing-2020-11-05-1949-nautilus-distro-basic-smithi/5593950/