2021-09-30T18:56:58.449 INFO:root:teuthology version: 1.1.0-f08b273e
2021-09-30T18:56:58.450 INFO:teuthology.results:seed: '5264'
2021-09-30T18:57:40.284 INFO:root:teuthology version: 1.1.0-f08b273e
2021-09-30T18:57:40.287 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-09-30T19:24:41.832 INFO:teuthology.results:Tests finished! gathering results...
2021-09-30T19:24:41.940 INFO:scrape:Found 2 jobs
2021-09-30T19:24:42.049 INFO:scrape:Found 1 distinct failure reasons
2021-09-30T19:24:42.050 INFO:scrape:Failure: Command failed on smithi018 with status 4: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9c76f6fa-2222-11ec-8c25-001a4aab830c -- bash -c \'ceph fs dump | jq -e ".filesystems | length == 1"\''
2021-09-30T19:24:42.051 INFO:scrape:2 jobs: ['6414894', '6414895']
2021-09-30T19:24:42.051 INFO:scrape:suites intersection: ['1-volume/{0-create', '2-client', '2-verify}', '3-upgrade-with-workload', '4-verify}}', 'centos_8.3_container_tools_3.0', 'conf/{client', 'fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'roles', 'tasks/{0-v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2021-09-30T19:24:42.052 INFO:scrape:suites union: ['1-ranks/1', '1-ranks/2', '1-volume/{0-create', '2-client', '2-verify}', '3-upgrade-with-workload', '4-verify}}', 'centos_8.3_container_tools_3.0', 'conf/{client', 'fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'roles', 'tasks/{0-v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2021-09-30T19:24:42.052 INFO:scrape:
2021-09-30T19:24:42.053 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2021-09-30_18:56:52-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/pdonnell-2021-09-30_18:56:52-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/pdonnell-2021-09-30_18:56:52-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/
failed:  2
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[6414894]  fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:15:40
info:   http://pulpito.ceph.com/pdonnell-2021-09-30_18:56:52-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/6414894/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2021-09-30_18:56:52-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/6414894/
sentry: https://sentry.ceph.com/organizations/ceph/?query=272715bb890d4f24b72c72a281d39137

    Command failed on smithi018 with status 4: 'sudo
    /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    9c76f6fa-2222-11ec-8c25-001a4aab830c -- bash -c \'ceph fs dump | jq -e
    ".filesystems | length == 1"\''

[6414895]  fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:15:54
info:   http://pulpito.ceph.com/pdonnell-2021-09-30_18:56:52-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/6414895/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2021-09-30_18:56:52-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/6414895/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7e73112da61a43328c2b9ab9b4b18d50

    Command failed on smithi022 with status 4: 'sudo
    /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    e20127a4-2222-11ec-8c25-001a4aab830c -- bash -c \'ceph fs dump | jq -e
    ".filesystems | length == 1"\''