2021-02-19T06:07:51.292 INFO:root:teuthology version: 1.1.0-9a6b849c 2021-02-19T06:07:51.294 INFO:teuthology.results:subset: '3/4' 2021-02-19T06:07:51.295 INFO:teuthology.results:seed: '6262' 2021-02-19T06:09:15.136 INFO:root:teuthology version: 1.1.0-9a6b849c 2021-02-19T06:09:15.160 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-02-19T06:24:15.517 INFO:teuthology.results:Tests finished! gathering results... 2021-02-19T06:24:17.767 INFO:scrape:Found 10 jobs 2021-02-19T06:24:27.714 INFO:scrape:Found 2 distinct failure reasons 2021-02-19T06:24:27.720 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=wip-vshankar-cephfs-mirror-valgrind-20210219-101103 2021-02-19T06:24:27.721 INFO:scrape:5 jobs: ['5894329', '5894332', '5894330', '5894328', '5894331'] 2021-02-19T06:24:27.721 INFO:scrape: 2021-02-19T06:24:27.721 INFO:scrape:Failure: Command failed on smithi097 with status 1: 'sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ceph-base-debuginfo ceph-common-debuginfo ceph-immutable-object-cache-debuginfo ceph-radosgw-debuginfo ceph-test-debuginfo ceph-base-debuginfo ceph-mgr-debuginfo ceph-mds-debuginfo ceph-mon-debuginfo ceph-osd-debuginfo ceph-fuse-debuginfo librados-devel-debuginfo libcephfs2-debuginfo librados2-debuginfo librbd1-debuginfo python3-cephfs-debuginfo python3-rados-debuginfo python3-rbd-debuginfo python3-rgw-debuginfo rbd-fuse-debuginfo rbd-mirror-debuginfo rbd-nbd-debuginfo python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel' 2021-02-19T06:24:27.722 INFO:scrape:5 jobs: ['5894333', '5894337', '5894334', '5894335', '5894336'] 2021-02-19T06:24:27.722 INFO:scrape:suites intersection: ['centos_latest', 'clients/mirror', 'cluster/1-node', 'fs/valgrind/{begin', 'mirror/{cephfs-mirror/one-per-cluster', 'mount/fuse', 'notcmalloc}', 'overrides/whitelist_health', 'tasks/mirror}'] 2021-02-19T06:24:27.723 INFO:scrape:suites union: ['centos_latest', 'clients/mirror', 'cluster/1-node', 'fs/valgrind/{begin', 'mirror/{cephfs-mirror/one-per-cluster', 'mount/fuse', 'notcmalloc}', 'overrides/whitelist_health', 'tasks/mirror}'] 2021-02-19T06:24:27.723 INFO:scrape: 2021-02-19T06:24:27.724 INFO:teuthology.results:Sending results to vshankar@redhat.com: Test Run: vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/ failed: 10 dead: 0 running: 0 waiting: 0 queued: 0 passed: 0 Fail ================================================================= [5894328] fs/mirror/{begin cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{ubuntu_latest} tasks/mirror} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894328/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894328/ sentry: https://sentry.ceph.com/organizations/ceph/?query=89660d071daf41a89ef95fb56bb80892 Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref= wip-vshankar-cephfs-mirror-valgrind-20210219-101103 [5894329] fs/mirror/{begin cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{ubuntu_latest} tasks/mirror} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894329/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894329/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3e6249cef68c432ab6209c1de776df3f Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref= wip-vshankar-cephfs-mirror-valgrind-20210219-101103 [5894330] fs/mirror/{begin cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{ubuntu_latest} tasks/mirror} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894330/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894330/ sentry: https://sentry.ceph.com/organizations/ceph/?query=045f152cd0aa46bab10c7d5e9203666e Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref= wip-vshankar-cephfs-mirror-valgrind-20210219-101103 [5894331] fs/mirror/{begin cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{ubuntu_latest} tasks/mirror} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894331/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894331/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f3247fd4abd54ef89217e9be6f6d312e Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref= wip-vshankar-cephfs-mirror-valgrind-20210219-101103 [5894332] fs/mirror/{begin cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{ubuntu_latest} tasks/mirror} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894332/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894332/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9ec459ecb73e4177867a60134efdf4d9 Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref= wip-vshankar-cephfs-mirror-valgrind-20210219-101103 [5894333] fs/valgrind/{begin centos_latest mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror} notcmalloc} ----------------------------------------------------------------- time: 00:05:21 info: http://pulpito.ceph.com/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894333/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894333/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4c9279eaf2924d3984804e2afaaa0d71 Command failed on smithi097 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd- mirror rbd-nbd ceph-base-debuginfo ceph-common-debuginfo ceph-immutable- object-cache-debuginfo ceph-radosgw-debuginfo ceph-test-debuginfo ceph- base-debuginfo ceph-mgr-debuginfo ceph-mds-debuginfo ceph-mon-debuginfo ceph-osd-debuginfo ceph-fuse-debuginfo librados-devel-debuginfo libcephfs2-debuginfo librados2-debuginfo librbd1-debuginfo python3-cephfs- debuginfo python3-rados-debuginfo python3-rbd-debuginfo python3-rgw- debuginfo rbd-fuse-debuginfo rbd-mirror-debuginfo rbd-nbd-debuginfo python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel' [5894334] fs/valgrind/{begin centos_latest mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror} notcmalloc} ----------------------------------------------------------------- time: 00:05:29 info: http://pulpito.ceph.com/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894334/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894334/ sentry: https://sentry.ceph.com/organizations/ceph/?query=53ee84635971467fbabe072a0d5dd688 Command failed on smithi012 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd- mirror rbd-nbd ceph-base-debuginfo ceph-common-debuginfo ceph-immutable- object-cache-debuginfo ceph-radosgw-debuginfo ceph-test-debuginfo ceph- base-debuginfo ceph-mgr-debuginfo ceph-mds-debuginfo ceph-mon-debuginfo ceph-osd-debuginfo ceph-fuse-debuginfo librados-devel-debuginfo libcephfs2-debuginfo librados2-debuginfo librbd1-debuginfo python3-cephfs- debuginfo python3-rados-debuginfo python3-rbd-debuginfo python3-rgw- debuginfo rbd-fuse-debuginfo rbd-mirror-debuginfo rbd-nbd-debuginfo python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel' [5894335] fs/valgrind/{begin centos_latest mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror} notcmalloc} ----------------------------------------------------------------- time: 00:05:21 info: http://pulpito.ceph.com/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894335/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894335/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a8de64855c78434a87a5e0e9782da00d Command failed on smithi187 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd- mirror rbd-nbd ceph-base-debuginfo ceph-common-debuginfo ceph-immutable- object-cache-debuginfo ceph-radosgw-debuginfo ceph-test-debuginfo ceph- base-debuginfo ceph-mgr-debuginfo ceph-mds-debuginfo ceph-mon-debuginfo ceph-osd-debuginfo ceph-fuse-debuginfo librados-devel-debuginfo libcephfs2-debuginfo librados2-debuginfo librbd1-debuginfo python3-cephfs- debuginfo python3-rados-debuginfo python3-rbd-debuginfo python3-rgw- debuginfo rbd-fuse-debuginfo rbd-mirror-debuginfo rbd-nbd-debuginfo python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel' [5894336] fs/valgrind/{begin centos_latest mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror} notcmalloc} ----------------------------------------------------------------- time: 00:05:24 info: http://pulpito.ceph.com/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894336/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894336/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f8d3f92ddc6a4ffc8066d679d8367a3c Command failed on smithi080 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd- mirror rbd-nbd ceph-base-debuginfo ceph-common-debuginfo ceph-immutable- object-cache-debuginfo ceph-radosgw-debuginfo ceph-test-debuginfo ceph- base-debuginfo ceph-mgr-debuginfo ceph-mds-debuginfo ceph-mon-debuginfo ceph-osd-debuginfo ceph-fuse-debuginfo librados-devel-debuginfo libcephfs2-debuginfo librados2-debuginfo librbd1-debuginfo python3-cephfs- debuginfo python3-rados-debuginfo python3-rbd-debuginfo python3-rgw- debuginfo rbd-fuse-debuginfo rbd-mirror-debuginfo rbd-nbd-debuginfo python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel' [5894337] fs/valgrind/{begin centos_latest mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror} notcmalloc} ----------------------------------------------------------------- time: 00:05:33 info: http://pulpito.ceph.com/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894337/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2021-02-19_06:07:35-fs-wip-vshankar-cephfs-mirror-valgrind-20210219-101103-distro-basic-smithi/5894337/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8b8632b4128845f49616f167e4e79a78 Command failed on smithi067 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd- mirror rbd-nbd ceph-base-debuginfo ceph-common-debuginfo ceph-immutable- object-cache-debuginfo ceph-radosgw-debuginfo ceph-test-debuginfo ceph- base-debuginfo ceph-mgr-debuginfo ceph-mds-debuginfo ceph-mon-debuginfo ceph-osd-debuginfo ceph-fuse-debuginfo librados-devel-debuginfo libcephfs2-debuginfo librados2-debuginfo librbd1-debuginfo python3-cephfs- debuginfo python3-rados-debuginfo python3-rbd-debuginfo python3-rgw- debuginfo rbd-fuse-debuginfo rbd-mirror-debuginfo rbd-nbd-debuginfo python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel'