2022-01-19T04:57:09.952 INFO:root:teuthology version: 1.1.0-3ff86a5d 2022-01-19T04:57:09.954 INFO:teuthology.results:subset: '7/32' 2022-01-19T04:57:09.954 INFO:teuthology.results:seed: '735' 2022-01-19T04:59:50.977 INFO:root:teuthology version: 1.1.0-3ff86a5d 2022-01-19T04:59:50.980 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-01-19T05:21:52.661 INFO:teuthology.results:Tests finished! gathering results... 2022-01-19T05:21:53.022 INFO:scrape:Found 5 jobs 2022-01-19T05:21:53.417 INFO:scrape:Found 2 distinct failure reasons 2022-01-19T05:21:53.418 INFO:scrape:Failure: Command failed on smithi165 with status 1: 'sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel' 2022-01-19T05:21:53.418 INFO:scrape:4 jobs: ['6626402', '6626401', '6626399', '6626398'] 2022-01-19T05:21:53.419 INFO:scrape:suites intersection: [] 2022-01-19T05:21:53.419 INFO:scrape:suites union: ['2-workunit/suites/pjd}}', 'cephfs-mirror/one-per-cluster', 'clients/{mirror}', 'cluster/{1-node}', 'clusters/1a3s-mds-2c-client', 'clusters/1a3s-mds-4c-client', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8.stream}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'fs/mirror/{begin', 'fs/mixed-clients/{begin', 'fs/thrash/workloads/{begin', 'kclient-overrides/{distro/stock/{k-stock', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'overrides/{no_client_pidfile', 'overrides/{osd-asserts', 'overrides/{whitelist_health}', 'prefetch_dirfrags/yes', 'races', 'ranks/1', 'rhel_8}', 'session_timeout', 'supported-random-distros$/{rhel_8}', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}', 'tasks/mirror}', 'tasks/scrub}', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-01-19T05:21:53.420 INFO:scrape: 2022-01-19T05:21:53.420 INFO:scrape:Failure: Command failed on smithi003 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:e19ff274e83a4c1cdfbf4679d70bc1a6fde36f6e pull' 2022-01-19T05:21:53.421 INFO:scrape:1 jobs: ['6626400'] 2022-01-19T05:21:53.421 INFO:scrape:suites: ['1-mount', '2-workload/dbench', 'distro/centos_latest}', 'fs/cephadm/multivolume/{0-start'] 2022-01-19T05:21:53.422 INFO:scrape: 2022-01-19T05:21:53.424 INFO:teuthology.results:Sending results to vshankar@redhat.com: Test Run: vshankar-2022-01-19_04:57:04-fs-wip-vshankar-testing-20220113-121125-testing-default-smithi ================================================================= info: http://pulpito.ceph.com/vshankar-2022-01-19_04:57:04-fs-wip-vshankar-testing-20220113-121125-testing-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/vshankar-2022-01-19_04:57:04-fs-wip-vshankar-testing-20220113-121125-testing-default-smithi/ failed: 5 dead: 0 running: 0 waiting: 0 queued: 0 passed: 0 Fail ================================================================= [6626398] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/scrub} ----------------------------------------------------------------- time: 00:09:51 info: http://pulpito.ceph.com/vshankar-2022-01-19_04:57:04-fs-wip-vshankar-testing-20220113-121125-testing-default-smithi/6626398/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-01-19_04:57:04-fs-wip-vshankar-testing-20220113-121125-testing-default-smithi/6626398/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e15a927b9bd047eb9acfb8dc651dcac6 Command failed on smithi096 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd- fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util- linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util- linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util- linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel' [6626399] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:09:40 info: http://pulpito.ceph.com/vshankar-2022-01-19_04:57:04-fs-wip-vshankar-testing-20220113-121125-testing-default-smithi/6626399/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-01-19_04:57:04-fs-wip-vshankar-testing-20220113-121125-testing-default-smithi/6626399/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d2b22609e1c84096b8a6f0703945388c Command failed on smithi159 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd- fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util- linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util- linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util- linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel' [6626400] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/centos_latest} ----------------------------------------------------------------- time: 00:11:57 info: http://pulpito.ceph.com/vshankar-2022-01-19_04:57:04-fs-wip-vshankar-testing-20220113-121125-testing-default-smithi/6626400/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-01-19_04:57:04-fs-wip-vshankar-testing-20220113-121125-testing-default-smithi/6626400/ sentry: https://sentry.ceph.com/organizations/ceph/?query=14d46e6caed04e7181bbaf7e2e697e67 Command failed on smithi003 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:e19ff274e83a4c1cdfbf4679d70bc1a6fde36f6e pull' [6626401] fs/mirror/{begin cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{rhel_8} tasks/mirror} ----------------------------------------------------------------- time: 00:11:57 info: http://pulpito.ceph.com/vshankar-2022-01-19_04:57:04-fs-wip-vshankar-testing-20220113-121125-testing-default-smithi/6626401/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-01-19_04:57:04-fs-wip-vshankar-testing-20220113-121125-testing-default-smithi/6626401/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7607ea7aed7044edacb2dcf63d82409d Command failed on smithi189 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd- fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util- linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel' [6626402] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:09:38 info: http://pulpito.ceph.com/vshankar-2022-01-19_04:57:04-fs-wip-vshankar-testing-20220113-121125-testing-default-smithi/6626402/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2022-01-19_04:57:04-fs-wip-vshankar-testing-20220113-121125-testing-default-smithi/6626402/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8ad867f78081450faa60eb8de28e98b2 Command failed on smithi165 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd- fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util- linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util- linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util- linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel'