2021-05-24T09:13:32.866 INFO:root:teuthology version: 1.1.0-d92ec140
2021-05-24T09:13:32.868 INFO:teuthology.results:seed: '8250'
2021-05-24T09:53:16.726 INFO:root:teuthology version: 1.1.0-d92ec140
2021-05-24T09:53:16.760 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-05-24T11:24:22.485 INFO:teuthology.results:Tests finished! gathering results...
2021-05-24T11:24:25.172 INFO:scrape:Found 48 jobs
2021-05-24T11:24:27.211 INFO:scrape:Found 3 distinct failure reasons
2021-05-24T11:24:27.212 INFO:scrape:Failure: Command failed (s3 tests against rgw) on smithi003 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!test_of_sts,!webidentity_test,!fails_with_subdomain'"
2021-05-24T11:24:27.212 INFO:scrape:2 jobs: ['6131378', '6131382']
2021-05-24T11:24:27.213 INFO:scrape:suites intersection: ['centos_latest', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'ragweed', 'reshard', 'rgw:verify/{0-install', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'tasks/{cls', 'validater/valgrind}']
2021-05-24T11:24:27.213 INFO:scrape:suites union: ['centos_latest', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw:verify/{0-install', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'tasks/{cls', 'validater/valgrind}']
2021-05-24T11:24:27.214 INFO:scrape:
2021-05-24T11:24:27.214 INFO:scrape:Failure: "2021-05-24T09:40:10.200493+0000 osd.6 (osd.6) 3 : cluster [WRN] slow request osd_op(client.4578.0:3693 56.10 56:0b0dc0e2:::245020db-64b9-44fc-a799-e7ebdb7da91a.4584.23__multipart_key1.2~FjtGYzduW3b_t_Rp68blapj0VsC3Rls.meta:head [create,call rgw.obj_remove in=40b,writefull 0~27 in=27b,setxattr user.rgw.acl (203) in=215b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.x-amz-content-sha256 (65) in=94b,setxattr user.rgw.x-amz-date (17) in=36b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e204) initiated 2021-05-24T09:39:39.916764+0000 currently delayed" in cluster log
2021-05-24T11:24:27.215 INFO:scrape:1 jobs: ['6131365']
2021-05-24T11:24:27.215 INFO:scrape:suites: ['centos_latest', 'clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw:verify/{0-install', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{cls', 'validater/lockdep}']
2021-05-24T11:24:27.215 INFO:scrape:
2021-05-24T11:24:27.216 INFO:scrape:Failure: Command failed (ragweed tests against rgw) on smithi073 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"
2021-05-24T11:24:27.216 INFO:scrape:3 jobs: ['6131372', '6131370', '6131376']
2021-05-24T11:24:27.217 INFO:scrape:suites intersection: ['centos_latest', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'overrides', 'ragweed', 'reshard', 'rgw:verify/{0-install', 's3tests-java', 's3tests}', 'striping$/{stripe-equals-chunk}', 'tasks/{cls', 'validater/valgrind}']
2021-05-24T11:24:27.217 INFO:scrape:suites union: ['centos_latest', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw:verify/{0-install', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'tasks/{cls', 'validater/valgrind}']
2021-05-24T11:24:27.218 INFO:scrape:
2021-05-24T11:24:27.221 INFO:teuthology.results:Sending results to mkogan@redhat.com: Test Run: mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/
failed:  6
dead:    0
running: 0
waiting: 0
queued:  0
passed:  42


Fail
=================================================================
[6131365]  rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:39:28
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131365/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131365/

    "2021-05-24T09:40:10.200493+0000 osd.6 (osd.6) 3 : cluster [WRN] slow
    request osd_op(client.4578.0:3693 56.10 56:0b0dc0e2:::245020db-64b9-44fc-a7
    99-e7ebdb7da91a.4584.23__multipart_key1.2~FjtGYzduW3b_t_Rp68blapj0VsC3Rls.m
    eta:head [create,call rgw.obj_remove in=40b,writefull 0~27 in=27b,setxattr
    user.rgw.acl (203) in=215b,setxattr user.rgw.content_type (25)
    in=46b,setxattr user.rgw.x-amz-content-sha256 (65) in=94b,setxattr
    user.rgw.x-amz-date (17) in=36b,call rgw.obj_store_pg_ver in=44b,setxattr
    user.rgw.source_zone (4) in=24b] snapc 0=[]
    ondisk+write+known_if_redirected e204) initiated
    2021-05-24T09:39:39.916764+0000 currently delayed" in cluster log

[6131370]  rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:23:37
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131370/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131370/
sentry: https://sentry.ceph.com/organizations/ceph/?query=27e35bebe77641deab4177dec740aeeb

    Command failed (ragweed tests against rgw) on smithi045 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[6131372]  rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:22:14
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131372/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131372/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3e375837d64546659523561139bc34e1

    Command failed (ragweed tests against rgw) on smithi073 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[6131376]  rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:25:23
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131376/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131376/
sentry: https://sentry.ceph.com/organizations/ceph/?query=be9852d055f44b749da50783c4c4f873

    Command failed (ragweed tests against rgw) on smithi018 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[6131378]  rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   01:29:43
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131378/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131378/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c296b368d5c74a69b3868d52fa5cd67a

    Command failed (s3 tests against rgw) on smithi003 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!test_of_sts,!webidentity_test,!fails_with_subdomain'"

[6131382]  rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   01:15:22
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131382/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131382/
sentry: https://sentry.ceph.com/organizations/ceph/?query=597486412d8d4300ae3edd93dfd1d9c0

    Command failed (s3 tests against rgw) on smithi094 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!test_of_sts,!webidentity_test,!fails_with_subdomain'"



Pass
=================================================================
[6131363] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:30:22
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131363/

[6131364] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:25:07
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131364/

[6131366] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:28:04
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131366/

[6131367] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:27:35
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131367/

[6131368] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:23:41
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131368/

[6131369] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:29:41
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131369/

[6131371] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:31:47
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131371/

[6131373] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:31:44
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131373/

[6131374] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:13:36
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131374/

[6131375] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:27:16
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131375/

[6131377] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:27:49
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131377/

[6131379] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:28:38
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131379/

[6131380] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:28:24
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131380/

[6131381] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:29:33
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131381/

[6131383] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:28:51
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131383/

[6131384] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:19:00
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131384/

[6131385] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:28:58
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131385/

[6131386] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:17:45
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131386/

[6131387] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:29:44
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131387/

[6131388] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:22:45
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131388/

[6131389] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:29:23
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131389/

[6131390] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:22:08
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131390/

[6131391] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:32:06
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131391/

[6131392] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:27:33
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131392/

[6131393] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:31:16
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131393/

[6131394] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:12:38
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131394/

[6131395] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:30:03
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131395/

[6131396] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:12:25
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131396/

[6131397] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:29:13
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131397/

[6131398] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:15:47
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131398/

[6131399] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:27:28
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131399/

[6131400] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:25:51
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131400/

[6131401] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:26:45
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131401/

[6131402] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:27:55
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131402/

[6131403] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:27:13
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131403/

[6131404] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:22:18
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131404/

[6131405] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:27:53
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131405/

[6131406] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:15:28
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131406/

[6131407] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:28:55
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131407/

[6131408] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:15:01
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131408/

[6131409] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:27:19
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131409/

[6131410] rgw:verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:15:22
info:   http://pulpito.ceph.com/mkogan-2021-05-24_09:13:18-rgw:verify-wip-rgw-d3n-cache-upstreaming_n02-distro-basic-smithi/6131410/