2020-05-20T05:55:13.096 INFO:root:teuthology version: 1.0.0-f695474 2020-05-20T05:55:13.098 INFO:teuthology.results:seed: '9800' 2020-05-20T06:25:23.465 INFO:root:teuthology version: 1.0.0-f695474 2020-05-20T06:25:23.490 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-05-20T07:18:24.722 INFO:teuthology.results:Tests finished! gathering results... 2020-05-20T07:18:24.817 INFO:teuthology.results:Sending results to gsalomon@redhat.com: Test Run: gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/ failed: 22 dead: 0 running: 0 waiting: 0 queued: 0 passed: 26 Fail ================================================================= [5071856] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:29:50 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071856/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071856/ "2020-05-20T06:14:10.919821+0000 mon.b (mon.0) 471 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5071860] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:28:47 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071860/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071860/ "2020-05-20T06:09:43.662467+0000 mon.a (mon.0) 245 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY)" in cluster log [5071862] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:28:32 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071862/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071862/ "2020-05-20T06:17:42.904197+0000 mon.a (mon.0) 452 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5071864] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:28:05 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071864/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071864/ "2020-05-20T06:14:50.179095+0000 mon.a (mon.0) 181 : cluster [WRN] Health check failed: Reduced data availability: 3 pgs inactive, 3 pgs peering (PG_AVAILABILITY)" in cluster log [5071866] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:34:51 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071866/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071866/ "2020-05-20T06:22:49.915603+0000 mon.a (mon.0) 390 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5071869] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:31:42 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071869/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071869/ "2020-05-20T06:28:17.726369+0000 mon.a (mon.0) 450 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5071871] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:30:04 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071871/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071871/ "2020-05-20T06:25:16.001260+0000 mon.b (mon.0) 461 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY)" in cluster log [5071873] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:28:46 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071873/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071873/ "2020-05-20T06:18:38.502388+0000 mon.a (mon.0) 255 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs inactive, 4 pgs peering (PG_AVAILABILITY)" in cluster log [5071875] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:26:49 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071875/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071875/ "2020-05-20T06:21:59.866009+0000 mon.b (mon.0) 197 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5071877] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:28:31 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071877/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071877/ "2020-05-20T06:24:02.966532+0000 mon.a (mon.0) 389 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5071878] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:28:03 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071878/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071878/ "2020-05-20T06:25:03.769432+0000 mon.b (mon.0) 192 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs inactive, 4 pgs peering (PG_AVAILABILITY)" in cluster log [5071879] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:27:20 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071879/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071879/ "2020-05-20T06:25:28.611090+0000 mon.b (mon.0) 217 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5071882] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:28:13 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071882/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071882/ "2020-05-20T06:23:25.949316+0000 mon.a (mon.0) 252 : cluster [WRN] Health check failed: Reduced data availability: 5 pgs inactive, 5 pgs peering (PG_AVAILABILITY)" in cluster log [5071884] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:27:40 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071884/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071884/ "2020-05-20T06:23:07.714897+0000 mon.b (mon.0) 198 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5071886] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:29:32 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071886/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071886/ "2020-05-20T06:32:33.896178+0000 mon.a (mon.0) 438 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [5071888] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:29:34 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071888/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071888/ "2020-05-20T06:29:13.825475+0000 mon.a (mon.0) 510 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5071893] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:33:25 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071893/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071893/ "2020-05-20T06:31:44.271718+0000 mon.a (mon.0) 327 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5071895] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:28:59 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071895/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071895/ "2020-05-20T06:34:19.603446+0000 mon.a (mon.0) 180 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5071899] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:29:11 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071899/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071899/ "2020-05-20T06:46:50.307511+0000 mon.b (mon.0) 208 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5071901] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:26:47 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071901/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071901/ "2020-05-20T06:35:41.491533+0000 mon.b (mon.0) 168 : cluster [WRN] Health check failed: Reduced data availability: 5 pgs inactive, 5 pgs peering (PG_AVAILABILITY)" in cluster log [5071902] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:30:36 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071902/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071902/ "2020-05-20T06:54:02.935874+0000 mon.b (mon.0) 178 : cluster [WRN] Health check failed: Reduced data availability: 8 pgs inactive, 8 pgs peering (PG_AVAILABILITY)" in cluster log [5071903] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:27:49 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071903/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071903/ "2020-05-20T06:46:02.969669+0000 mon.a (mon.0) 169 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs inactive, 4 pgs peering (PG_AVAILABILITY)" in cluster log Pass ================================================================= [5071857] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:33:58 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071857/ [5071858] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:26:27 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071858/ [5071859] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:30:30 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071859/ [5071861] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:28:46 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071861/ [5071863] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:27:01 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071863/ [5071865] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:27:01 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071865/ [5071867] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:31:28 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071867/ [5071868] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:31:51 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071868/ [5071870] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:26:51 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071870/ [5071872] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:27:11 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071872/ [5071874] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:27:17 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071874/ [5071876] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:28:04 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071876/ [5071880] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:27:52 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071880/ [5071881] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:28:27 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071881/ [5071883] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:27:56 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071883/ [5071885] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:26:57 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071885/ [5071887] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:27:33 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071887/ [5071889] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:26:20 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071889/ [5071890] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:30:13 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071890/ [5071891] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:27:41 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071891/ [5071892] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:26:31 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071892/ [5071894] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:26:22 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071894/ [5071896] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:26:36 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071896/ [5071897] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:28:32 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071897/ [5071898] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:27:57 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071898/ [5071900] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:28:53 info: http://pulpito.ceph.com/gsalomon-2020-05-20_05:54:07-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5071900/ 2020-05-20T07:18:24.918 INFO:teuthology.results:starting coverage generation 2020-05-20T07:18:24.924 ERROR:teuthology.results:error generating memo/results Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 35, in main int(args['--timeout']), args['--dry-run']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 84, in results generate_coverage(archive_dir, name) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 107, in generate_coverage archive_dir, File "/usr/lib/python3.6/subprocess.py", line 729, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage': '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage'