2023-07-11T20:33:41.702 INFO:root:teuthology version: 0.0.1.dev140+gb53f236
2023-07-11T20:33:41.703 INFO:teuthology.results:subset: '499/512'
2023-07-11T20:33:41.703 INFO:teuthology.results:seed: '420'
2023-07-11T20:34:32.627 INFO:root:teuthology version: 0.0.1.dev140+gb53f236
2023-07-11T20:34:32.630 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-07-11T21:15:35.773 INFO:teuthology.results:Tests finished! gathering results...
2023-07-11T21:15:36.115 INFO:scrape:Found 10 jobs
2023-07-11T21:15:36.289 INFO:scrape:Found 2 distinct failure reasons
2023-07-11T21:15:36.290 INFO:scrape:Failure: Command failed on smithi104 with status 1: 'sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel'
2023-07-11T21:15:36.290 INFO:scrape:1 jobs: ['7334203']
2023-07-11T21:15:36.291 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/strays}']
2023-07-11T21:15:36.291 INFO:scrape:
2023-07-11T21:15:36.291 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=e1900c40c6e3837d137496ba798ab6f9035488fd
2023-07-11T21:15:36.292 INFO:scrape:5 jobs: ['7334201', '7334200', '7334198', '7334199', '7334202']
2023-07-11T21:15:36.292 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/strays}']
2023-07-11T21:15:36.292 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/strays}']
2023-07-11T21:15:36.292 INFO:scrape:
2023-07-11T21:15:36.293 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/
failed:  6
dead:    0
running: 0
waiting: 0
queued:  0
passed:  4


Fail
=================================================================
[7334198]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays}
-----------------------------------------------------------------
time:   00:14:34
info:   http://pulpito.ceph.com/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334198/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334198/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5bcb3f286d5644d5b81661d20c24f34f

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=e19
    00c40c6e3837d137496ba798ab6f9035488fd

[7334199]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays}
-----------------------------------------------------------------
time:   00:14:54
info:   http://pulpito.ceph.com/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334199/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334199/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5be7cdb83af741c78e51badb7617346b

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=e19
    00c40c6e3837d137496ba798ab6f9035488fd

[7334200]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays}
-----------------------------------------------------------------
time:   00:18:04
info:   http://pulpito.ceph.com/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334200/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334200/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5b299e7b14404e09b537e2fa75e72d4f

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=e19
    00c40c6e3837d137496ba798ab6f9035488fd

[7334201]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays}
-----------------------------------------------------------------
time:   00:12:22
info:   http://pulpito.ceph.com/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334201/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334201/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1d4bea69486d4d72b91779a6a4253ac7

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=e19
    00c40c6e3837d137496ba798ab6f9035488fd

[7334202]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays}
-----------------------------------------------------------------
time:   00:14:28
info:   http://pulpito.ceph.com/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334202/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334202/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6db0562531ca458985b00554998a6305

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=e19
    00c40c6e3837d137496ba798ab6f9035488fd

[7334203]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays}
-----------------------------------------------------------------
time:   00:13:37
info:   http://pulpito.ceph.com/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334203/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334203/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2f7fab1d30f347b1bf3c55cf67e343eb

    Command failed on smithi104 with status 1: 'sudo yum -y install ceph-
    radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-
    mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-
    mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel
    librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-
    fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex
    elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux
    libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump
    xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel
    bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-
    linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump
    xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel
    bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-
    linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump
    xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel
    bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-
    linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump
    xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel'



Pass
=================================================================
[7334204] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays}
time:   00:30:58
info:   http://pulpito.ceph.com/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334204/

[7334205] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays}
time:   00:31:23
info:   http://pulpito.ceph.com/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334205/

[7334206] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays}
time:   00:31:07
info:   http://pulpito.ceph.com/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334206/

[7334207] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays}
time:   00:31:10
info:   http://pulpito.ceph.com/pdonnell-2023-07-11_20:33:37-fs:functional-quincy_release-distro-default-smithi/7334207/