2021-09-30T17:31:26.819 INFO:root:teuthology version: 1.1.0-f08b273e
2021-09-30T17:31:26.820 INFO:teuthology.results:seed: '361'
2021-09-30T17:32:09.684 INFO:root:teuthology version: 1.1.0-f08b273e
2021-09-30T17:32:09.690 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-09-30T17:59:11.403 INFO:teuthology.results:Tests finished! gathering results...
2021-09-30T17:59:11.490 INFO:scrape:Found 2 jobs
2021-09-30T17:59:11.593 INFO:scrape:Found 1 distinct failure reasons
2021-09-30T17:59:11.594 INFO:scrape:Failure: Command failed on smithi042 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c3438f84-2216-11ec-8c25-001a4aab830c -- bash -c \'ceph fs dump | jq -e \'"\'"\'.filesystems | length == 1\''
2021-09-30T17:59:11.594 INFO:scrape:2 jobs: ['6414887', '6414886']
2021-09-30T17:59:11.595 INFO:scrape:suites intersection: ['1-volume/{0-create', '2-client', '2-verify}', '3-upgrade-with-workload', '4-verify}}', 'centos_8.3_container_tools_3.0', 'conf/{client', 'fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'roles', 'tasks/{0-v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2021-09-30T17:59:11.595 INFO:scrape:suites union: ['1-ranks/1', '1-ranks/2', '1-volume/{0-create', '2-client', '2-verify}', '3-upgrade-with-workload', '4-verify}}', 'centos_8.3_container_tools_3.0', 'conf/{client', 'fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'roles', 'tasks/{0-v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2021-09-30T17:59:11.596 INFO:scrape:
2021-09-30T17:59:11.596 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2021-09-30_17:31:21-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/pdonnell-2021-09-30_17:31:21-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/pdonnell-2021-09-30_17:31:21-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/
failed:  2
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[6414886]  fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:15:59
info:   http://pulpito.ceph.com/pdonnell-2021-09-30_17:31:21-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/6414886/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2021-09-30_17:31:21-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/6414886/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8abc5c8b386b403b916245978fcd479c

    Command failed on smithi037 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    f07df232-2216-11ec-8c25-001a4aab830c -- bash -c \'ceph fs dump | jq -e
    \'"\'"\'.filesystems | length == 1\''

[6414887]  fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:16:05
info:   http://pulpito.ceph.com/pdonnell-2021-09-30_17:31:21-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/6414887/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2021-09-30_17:31:21-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/6414887/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d008d280937f4333adc3179f35dc0ef1

    Command failed on smithi042 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    c3438f84-2216-11ec-8c25-001a4aab830c -- bash -c \'ceph fs dump | jq -e
    \'"\'"\'.filesystems | length == 1\''