2020-08-20T20:52:59.593 INFO:root:teuthology version: 1.0.0-c0ecc52 2020-08-20T20:52:59.595 INFO:teuthology.results:subset: '859/9999' 2020-08-20T20:52:59.595 INFO:teuthology.results:seed: '2477' 2020-08-20T21:14:54.655 INFO:root:teuthology version: 1.0.0-c0ecc52 2020-08-20T21:14:54.673 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-08-21T00:28:59.177 INFO:teuthology.results:Tests finished! gathering results... 2020-08-21T00:29:07.990 INFO:scrape:Found 19 jobs 2020-08-21T00:29:10.322 INFO:scrape:Found 2 distinct failure reasons 2020-08-21T00:29:10.323 INFO:scrape:Failure: wait_for_clean: failed before timeout expired 2020-08-21T00:29:10.323 INFO:scrape:1 jobs: ['5361890'] 2020-08-21T00:29:10.323 INFO:scrape:suites: ['1-install/nautilus', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{rbd-cls', '5-workload/{radosbench', '6-finish-upgrade', '7-octopus', '8-workload/{rbd-python', 'bluestore-bitmap', 'rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack', 'rbd-import-export', 'rbd_api}', 'readwrite', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_latest}'] 2020-08-21T00:29:10.323 INFO:scrape: 2020-08-21T00:29:10.324 INFO:scrape:Failure: Test failure: test_cluster_set_reset_user_config (tasks.cephfs.test_nfs.TestNFS) 2020-08-21T00:29:10.324 INFO:scrape:4 jobs: ['5361876', '5361880', '5361885', '5361889'] 2020-08-21T00:29:10.324 INFO:scrape:suites intersection: ['task/test_orch_cli}'] 2020-08-21T00:29:10.324 INFO:scrape:suites union: ['rados/cephadm/workunits/{distro/centos_latest', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_orch_cli}'] 2020-08-21T00:29:10.325 INFO:scrape: 2020-08-21T00:29:10.327 INFO:teuthology.results:Sending results to tchaikov@gmail.com: Test Run: kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/ failed: 5 dead: 0 running: 0 waiting: 0 queued: 0 passed: 14 Fail ================================================================= [5361876] rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_orch_cli} ----------------------------------------------------------------- time: 00:17:16 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361876/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361876/ sentry: https://sentry.ceph.com/sepia/teuthology/?q=8a3df313355a4b9abc5d52bd0ef837eb Test failure: test_cluster_set_reset_user_config (tasks.cephfs.test_nfs.TestNFS) [5361880] rados/cephadm/workunits/{distro/centos_latest task/test_orch_cli} ----------------------------------------------------------------- time: 00:16:18 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361880/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361880/ sentry: https://sentry.ceph.com/sepia/teuthology/?q=be6ff7ec317943f7a6490704faf702e4 Test failure: test_cluster_set_reset_user_config (tasks.cephfs.test_nfs.TestNFS) [5361885] rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_orch_cli} ----------------------------------------------------------------- time: 00:17:32 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361885/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361885/ sentry: https://sentry.ceph.com/sepia/teuthology/?q=7d4667b009e64cf28dbc2b8d2d0fe711 Test failure: test_cluster_set_reset_user_config (tasks.cephfs.test_nfs.TestNFS) [5361889] rados/cephadm/workunits/{distro/centos_latest task/test_orch_cli} ----------------------------------------------------------------- time: 00:16:38 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361889/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361889/ sentry: https://sentry.ceph.com/sepia/teuthology/?q=f2abca2ada9c4cca8442c79cd035f626 Test failure: test_cluster_set_reset_user_config (tasks.cephfs.test_nfs.TestNFS) [5361890] rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-octopus 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap thrashosds-health ubuntu_latest} ----------------------------------------------------------------- time: 00:59:12 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361890/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361890/ sentry: https://sentry.ceph.com/sepia/teuthology/?q=d9c40bd859eb486980a38e38dd0abb32 wait_for_clean: failed before timeout expired Pass ================================================================= [5361874] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:27:52 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361874/ [5361875] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects-balanced} time: 00:21:15 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361875/ [5361877] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read} time: 00:21:11 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361877/ [5361878] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:26:49 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361878/ [5361879] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/osd-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:27:39 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361879/ [5361881] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/fastclose rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:23:25 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361881/ [5361882] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/osd-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:41:47 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361882/ [5361883] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:31 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361883/ [5361884] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-overwrites} time: 00:21:05 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361884/ [5361886] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:28:40 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361886/ [5361887] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/fastclose rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:31:49 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361887/ [5361888] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/fastclose objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-small-objects-balanced} time: 00:19:49 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361888/ [5361891] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-fast-read} time: 00:19:58 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361891/ [5361892] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:22:14 info: http://pulpito.ceph.com/kchai-2020-08-20_07:49:11-rados-master-distro-basic-smithi/5361892/