2020-11-03T11:47:10.985 INFO:root:teuthology version: 1.0.0-41b569f 2020-11-03T11:47:10.987 INFO:teuthology.results:subset: '311/9999' 2020-11-03T11:47:10.987 INFO:teuthology.results:seed: '8967' 2020-11-03T11:51:46.157 INFO:root:teuthology version: 1.0.0-41b569f 2020-11-03T11:51:46.395 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-11-03T13:40:48.267 INFO:teuthology.results:Tests finished! gathering results... 2020-11-03T13:40:49.453 INFO:scrape:Found 10 jobs 2020-11-03T13:40:49.458 INFO:scrape:Found 1 distinct failure reasons 2020-11-03T13:40:49.458 INFO:scrape:Timeout 120 running ceph --cluster ceph --log-early tell osd.0 flush_pg_stats 2020-11-03T13:40:49.459 INFO:scrape:1 jobs: ['5586799'] 2020-11-03T13:40:49.459 INFO:scrape:suites: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-partial-recovery}', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/crush-compat', 'msgr-failures/fastclose', 'msgr/async', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados:thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{rhel_latest}', 'thrashers/pggrow', 'thrashosds-health', 'workloads/pool-snaps-few-objects}'] 2020-11-03T13:40:49.459 INFO:scrape: 2020-11-03T13:40:49.460 INFO:teuthology.results:Sending results to dupadhya@redhat.com: Test Run: ideepika-2020-11-03_11:44:46-rados:thrash-wip-yuri8-testing-2020-11-03-1503-deepika-testing-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/ideepika-2020-11-03_11:44:46-rados:thrash-wip-yuri8-testing-2020-11-03-1503-deepika-testing-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/ideepika-2020-11-03_11:44:46-rados:thrash-wip-yuri8-testing-2020-11-03-1503-deepika-testing-distro-basic-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 9 Fail ================================================================= [5586799] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_latest} thrashers/pggrow thrashosds-health workloads/pool-snaps-few-objects} ----------------------------------------------------------------- time: 00:32:50 info: http://pulpito.ceph.com/ideepika-2020-11-03_11:44:46-rados:thrash-wip-yuri8-testing-2020-11-03-1503-deepika-testing-distro-basic-smithi/5586799/ log: http://qa-proxy.ceph.com/teuthology/ideepika-2020-11-03_11:44:46-rados:thrash-wip-yuri8-testing-2020-11-03-1503-deepika-testing-distro-basic-smithi/5586799/ Command failed on smithi203 with status 124: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 flush_pg_stats' Pass ================================================================= [5586797] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/off msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/cache} time: 00:09:44 info: http://pulpito.ceph.com/ideepika-2020-11-03_11:44:46-rados:thrash-wip-yuri8-testing-2020-11-03-1503-deepika-testing-distro-basic-smithi/5586797/ [5586798] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_latest} thrashers/none thrashosds-health workloads/dedup_tier} time: 00:12:26 info: http://pulpito.ceph.com/ideepika-2020-11-03_11:44:46-rados:thrash-wip-yuri8-testing-2020-11-03-1503-deepika-testing-distro-basic-smithi/5586798/ [5586800] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/off msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/rados_api_tests} time: 00:27:32 info: http://pulpito.ceph.com/ideepika-2020-11-03_11:44:46-rados:thrash-wip-yuri8-testing-2020-11-03-1503-deepika-testing-distro-basic-smithi/5586800/ [5586801] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/upmap msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/radosbench-high-concurrency} time: 00:38:55 info: http://pulpito.ceph.com/ideepika-2020-11-03_11:44:46-rados:thrash-wip-yuri8-testing-2020-11-03-1503-deepika-testing-distro-basic-smithi/5586801/ [5586802] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/radosbench} time: 01:34:05 info: http://pulpito.ceph.com/ideepika-2020-11-03_11:44:46-rados:thrash-wip-yuri8-testing-2020-11-03-1503-deepika-testing-distro-basic-smithi/5586802/ [5586803] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/off msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_latest} thrashers/morepggrow thrashosds-health workloads/redirect} time: 00:14:37 info: http://pulpito.ceph.com/ideepika-2020-11-03_11:44:46-rados:thrash-wip-yuri8-testing-2020-11-03-1503-deepika-testing-distro-basic-smithi/5586803/ [5586804] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/upmap msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/redirect_promote_tests} time: 00:09:52 info: http://pulpito.ceph.com/ideepika-2020-11-03_11:44:46-rados:thrash-wip-yuri8-testing-2020-11-03-1503-deepika-testing-distro-basic-smithi/5586804/ [5586805] rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/redirect_set_object} time: 00:11:00 info: http://pulpito.ceph.com/ideepika-2020-11-03_11:44:46-rados:thrash-wip-yuri8-testing-2020-11-03-1503-deepika-testing-distro-basic-smithi/5586805/ [5586806] rados:thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/off msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_latest} thrashers/careful thrashosds-health workloads/set-chunk-promote-flush} time: 00:13:41 info: http://pulpito.ceph.com/ideepika-2020-11-03_11:44:46-rados:thrash-wip-yuri8-testing-2020-11-03-1503-deepika-testing-distro-basic-smithi/5586806/