2021-07-09T18:55:13.296 INFO:root:teuthology version: 1.1.0-24e3280d
2021-07-09T18:55:13.297 INFO:teuthology.results:subset: '1/2'
2021-07-09T18:55:13.298 INFO:teuthology.results:seed: '2805'
2021-07-09T19:07:47.993 INFO:root:teuthology version: 1.1.0-24e3280d
2021-07-09T19:07:48.035 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-07-09T20:42:54.971 INFO:teuthology.results:Tests finished! gathering results...
2021-07-09T20:42:56.765 INFO:scrape:Found 24 jobs
2021-07-09T20:42:57.270 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261711/teuthology.log
2021-07-09T20:42:57.272 INFO:scrape:Found 2 distinct failure reasons
2021-07-09T20:42:57.272 INFO:scrape:Failure: "2021-07-09T19:26:26.205343+0000 mon.a (mon.0) 113 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log
2021-07-09T20:42:57.273 INFO:scrape:1 jobs: ['6261715']
2021-07-09T20:42:57.273 INFO:scrape:suites: ['centos_latest', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/filestore-xfs', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw:verify/{0-install', 'rgw_pool_type/ec', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'tasks/{cls', 'validater/lockdep}']
2021-07-09T20:42:57.273 INFO:scrape:
2021-07-09T20:42:57.274 INFO:scrape:Dead: None
2021-07-09T20:42:57.274 INFO:scrape:1 jobs: ['6261711']
2021-07-09T20:42:57.275 INFO:scrape:
2021-07-09T20:42:57.277 INFO:teuthology.results:Sending results to yehuda@redhat.com: Test Run: yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/
failed:  1
dead:    1
running: 0
waiting: 0
queued:  0
passed:  22


Fail
=================================================================
[6261715]  rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:30:52
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261715/
log:    http://qa-proxy.ceph.com/teuthology/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261715/

    "2021-07-09T19:26:26.205343+0000 mon.a (mon.0) 113 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log



Dead
=================================================================
[6261711]  rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261711/
log:    http://qa-proxy.ceph.com/teuthology/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261711/

    Error reimaging machines: reached maximum tries (60) after waiting for 900
    seconds



Pass
=================================================================
[6261693] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:30:53
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261693/

[6261694] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:21:44
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261694/

[6261695] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:28:47
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261695/

[6261696] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:27:07
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261696/

[6261697] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:29:14
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261697/

[6261698] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:24:41
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261698/

[6261699] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:31:17
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261699/

[6261700] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:12:19
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261700/

[6261701] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:31:29
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261701/

[6261702] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:15:01
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261702/

[6261703] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:31:54
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261703/

[6261704] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:13:54
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261704/

[6261705] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:28:36
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261705/

[6261706] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:27:09
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261706/

[6261707] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:29:27
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261707/

[6261708] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:27:51
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261708/

[6261709] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:29:23
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261709/

[6261710] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:23:38
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261710/

[6261712] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:13:34
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261712/

[6261713] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:30:38
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261713/

[6261714] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:17:18
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261714/

[6261716] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:16:48
info:   http://pulpito.ceph.com/yehuda-2021-07-09_11:54:40-rgw:verify-wip-rgw-realm-resolve-distro-basic-smithi/6261716/