2020-05-20T07:25:25.184 INFO:root:teuthology version: 1.0.0-f695474 2020-05-20T07:25:25.186 INFO:teuthology.results:seed: '1383' 2020-05-20T07:57:29.427 INFO:root:teuthology version: 1.0.0-f695474 2020-05-20T07:57:29.445 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-05-20T08:50:30.792 INFO:teuthology.results:Tests finished! gathering results... 2020-05-20T08:50:30.901 INFO:teuthology.results:Sending results to gsalomon@redhat.com: Test Run: gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/ failed: 24 dead: 0 running: 0 waiting: 0 queued: 0 passed: 24 Fail ================================================================= [5072274] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:27:30 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072274/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072274/ "2020-05-20T07:50:00.108996+0000 mon.a (mon.0) 386 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5072278] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:27:42 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072278/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072278/ "2020-05-20T08:15:49.254581+0000 mon.b (mon.0) 225 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5072279] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:28:58 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072279/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072279/ Command failed on smithi129 with status 1: 'mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log' [5072282] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:27:53 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072282/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072282/ "2020-05-20T08:08:58.288985+0000 mon.b (mon.0) 374 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5072284] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:28:26 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072284/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072284/ "2020-05-20T08:07:54.239974+0000 mon.b (mon.0) 304 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5072287] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:30:30 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072287/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072287/ "2020-05-20T07:48:38.838937+0000 mon.a (mon.0) 183 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY)" in cluster log [5072289] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:29:55 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072289/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072289/ "2020-05-20T08:01:12.458458+0000 mon.b (mon.0) 433 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5072290] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:27:41 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072290/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072290/ "2020-05-20T07:49:06.393242+0000 mon.b (mon.0) 169 : cluster [WRN] Health check failed: Reduced data availability: 5 pgs inactive, 5 pgs peering (PG_AVAILABILITY)" in cluster log [5072291] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:30:23 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072291/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072291/ "2020-05-20T07:50:12.922136+0000 mon.a (mon.0) 257 : cluster [WRN] Health check failed: Reduced data availability: 3 pgs inactive, 3 pgs peering (PG_AVAILABILITY)" in cluster log [5072293] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:28:54 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072293/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072293/ "2020-05-20T07:54:54.082879+0000 mon.a (mon.0) 221 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5072297] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:32:20 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072297/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072297/ "2020-05-20T08:00:26.097051+0000 mon.a (mon.0) 501 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5072298] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:28:47 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072298/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072298/ "2020-05-20T07:56:53.950209+0000 mon.a (mon.0) 469 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5072300] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:26:58 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072300/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072300/ "2020-05-20T08:25:29.190762+0000 mon.b (mon.0) 408 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY)" in cluster log [5072302] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:28:12 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072302/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072302/ "2020-05-20T07:53:51.611431+0000 mon.b (mon.0) 216 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5072304] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:31:07 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072304/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072304/ "2020-05-20T08:23:53.989744+0000 mon.a (mon.0) 176 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5072306] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:29:54 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072306/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072306/ "2020-05-20T08:09:45.569628+0000 mon.b (mon.0) 285 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5072308] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:29:16 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072308/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072308/ "2020-05-20T08:25:59.963326+0000 mon.b (mon.0) 429 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY)" in cluster log [5072311] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:29:59 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072311/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072311/ "2020-05-20T08:07:19.265549+0000 mon.b (mon.0) 170 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY)" in cluster log [5072313] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:28:41 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072313/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072313/ "2020-05-20T08:14:00.540405+0000 mon.a (mon.0) 441 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5072315] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:28:32 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072315/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072315/ "2020-05-20T08:10:04.233055+0000 mon.b (mon.0) 171 : cluster [WRN] Health check failed: Reduced data availability: 4 pgs inactive, 4 pgs peering (PG_AVAILABILITY)" in cluster log [5072317] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:29:08 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072317/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072317/ "2020-05-20T08:10:45.710878+0000 mon.a (mon.0) 229 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY)" in cluster log [5072319] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:27:59 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072319/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072319/ "2020-05-20T08:27:05.103053+0000 mon.a (mon.0) 170 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5072320] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:30:59 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072320/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072320/ "2020-05-20T08:27:24.865287+0000 mon.a (mon.0) 302 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY)" in cluster log [5072321] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:26:24 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072321/ log: http://qa-proxy.ceph.com/teuthology/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072321/ "2020-05-20T08:16:58.323671+0000 mon.b (mon.0) 217 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log Pass ================================================================= [5072275] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:28:49 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072275/ [5072276] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:27:49 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072276/ [5072277] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:30:58 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072277/ [5072280] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:30:18 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072280/ [5072281] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:31:53 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072281/ [5072283] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:25:48 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072283/ [5072285] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:26:51 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072285/ [5072286] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:27:31 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072286/ [5072288] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:25:43 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072288/ [5072292] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:28:22 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072292/ [5072294] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:26:57 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072294/ [5072295] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:26:48 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072295/ [5072296] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:27:35 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072296/ [5072299] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:33:27 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072299/ [5072301] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:27:44 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072301/ [5072303] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:26:42 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072303/ [5072305] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:27:07 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072305/ [5072307] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:26:49 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072307/ [5072309] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/valgrind.yaml} time: 00:26:25 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072309/ [5072310] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:27:09 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072310/ [5072312] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:26:51 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072312/ [5072314] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml sharding$/{default.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:27:07 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072314/ [5072316] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml sharding$/{single.yaml} striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:34:46 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072316/ [5072318] rgw:verify/{centos_latest.yaml clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml sharding$/{default.yaml} striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml reshard.yaml s3tests-java.yaml s3tests.yaml} validater/lockdep.yaml} time: 00:28:35 info: http://pulpito.ceph.com/gsalomon-2020-05-20_07:24:30-rgw:verify-s3select_integration_into_RGW_2-distro-basic-smithi/5072318/ 2020-05-20T08:50:31.269 INFO:teuthology.results:starting coverage generation 2020-05-20T08:50:31.287 ERROR:teuthology.results:error generating memo/results Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 35, in main int(args['--timeout']), args['--dry-run']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 84, in results generate_coverage(archive_dir, name) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 107, in generate_coverage archive_dir, File "/usr/lib/python3.6/subprocess.py", line 729, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage': '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage'