2021-09-30T19:26:09.691 INFO:root:teuthology version: 1.1.0-f08b273e 2021-09-30T19:26:09.693 INFO:teuthology.results:seed: '9081' 2021-09-30T19:26:51.058 INFO:root:teuthology version: 1.1.0-f08b273e 2021-09-30T19:26:51.063 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-09-30T20:09:53.865 INFO:teuthology.results:Tests finished! gathering results... 2021-09-30T20:09:54.034 INFO:scrape:Found 4 jobs 2021-09-30T20:09:55.496 INFO:scrape:Found 1 distinct failure reasons 2021-09-30T20:09:55.497 INFO:scrape:Failure: Command failed on smithi022 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9265bec-2226-11ec-8c25-001a4aab830c -- bash -c \'ceph --format=json fs dump | jq -e ".filesystems[0].mdsmap.max_mds == 2"\'' 2021-09-30T20:09:55.498 INFO:scrape:2 jobs: ['6414902', '6414905'] 2021-09-30T20:09:55.498 INFO:scrape:suites intersection: ['1-ranks/1', '1-volume/{0-create', '2-client', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.3_container_tools_3.0', 'conf/{client', 'fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'roles', 'tasks/{0-v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-09-30T20:09:55.499 INFO:scrape:suites union: ['1-ranks/1', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.3_container_tools_3.0', 'conf/{client', 'fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'roles', 'tasks/{0-v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-09-30T20:09:55.499 INFO:scrape: 2021-09-30T20:09:55.500 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2021-09-30_19:26:04-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/pdonnell-2021-09-30_19:26:04-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-09-30_19:26:04-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/ failed: 2 dead: 0 running: 0 waiting: 0 queued: 0 passed: 2 Fail ================================================================= [6414902] fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:31:49 info: http://pulpito.ceph.com/pdonnell-2021-09-30_19:26:04-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/6414902/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-09-30_19:26:04-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/6414902/ sentry: https://sentry.ceph.com/organizations/ceph/?query=28a940d913c14251a59c394962c93bf2 Command failed on smithi022 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c9265bec-2226-11ec-8c25-001a4aab830c -- bash -c \'ceph --format=json fs dump | jq -e ".filesystems[0].mdsmap.max_mds == 2"\'' [6414905] fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:32:33 info: http://pulpito.ceph.com/pdonnell-2021-09-30_19:26:04-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/6414905/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-09-30_19:26:04-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/6414905/ sentry: https://sentry.ceph.com/organizations/ceph/?query=57d8c0009de34be580a15db29f1ba69c Command failed on smithi096 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ad9112e6-2226-11ec-8c25-001a4aab830c -- bash -c \'ceph --format=json fs dump | jq -e ".filesystems[0].mdsmap.max_mds == 2"\'' Pass ================================================================= [6414903] fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:44 info: http://pulpito.ceph.com/pdonnell-2021-09-30_19:26:04-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/6414903/ [6414904] fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:33:26 info: http://pulpito.ceph.com/pdonnell-2021-09-30_19:26:04-fs:upgrade:mds_upgrade_sequence-wip-pdonnell-testing-20210930.003915-distro-basic-smithi/6414904/