2021-11-09T03:55:14.148 INFO:root:teuthology version: 1.1.0-27954452
2021-11-09T03:55:14.150 INFO:teuthology.results:seed: '5889'
2021-11-09T04:32:38.556 INFO:root:teuthology version: 1.1.0-27954452
2021-11-09T04:32:38.587 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-11-09T16:21:32.557 INFO:teuthology.results:Tests finished! gathering results...
2021-11-09T16:21:34.856 INFO:scrape:Found 20 jobs
2021-11-09T16:21:34.868 INFO:scrape:Found 1 distinct failure reasons
2021-11-09T16:21:34.870 INFO:scrape:Dead: b'2021-11-09T16:15:39.971 DEBUG:teuthology.exit:Finished running handlers'
2021-11-09T16:21:34.871 INFO:scrape:1 jobs: ['6493135']
2021-11-09T16:21:34.873 INFO:scrape:suites: ['1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.3_container_tools_3.0', 'conf/{client', 'fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'roles', 'tasks/{0-v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2021-11-09T16:21:34.873 INFO:scrape:
2021-11-09T16:21:34.875 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/
failed:  0
dead:    1
running: 0
waiting: 0
queued:  0
passed:  19


Dead
=================================================================
[6493135]  fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493135/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493135/

    hit max job timeout



Pass
=================================================================
[6493124] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:36:08
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493124/

[6493125] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:33:25
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493125/

[6493126] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:34:08
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493126/

[6493127] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:32:30
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493127/

[6493128] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:32:47
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493128/

[6493129] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:37:05
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493129/

[6493130] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:32:29
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493130/

[6493131] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:32:14
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493131/

[6493132] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:33:37
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493132/

[6493133] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:32:59
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493133/

[6493134] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:32:13
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493134/

[6493136] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:31:17
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493136/

[6493137] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:31:13
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493137/

[6493138] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:36:56
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493138/

[6493139] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:34:08
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493139/

[6493140] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:33:50
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493140/

[6493141] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:34:49
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493141/

[6493142] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:33:20
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493142/

[6493143] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:36:39
info:   http://pulpito.ceph.com/pdonnell-2021-11-09_03:54:01-fs:upgrade-wip-pdonnell-testing-20211109.015313-distro-basic-smithi/6493143/