2021-11-18T02:56:46.824 INFO:root:teuthology version: 1.1.0-6cfddd5e 2021-11-18T02:56:46.825 INFO:teuthology.results:subset: '30/32' 2021-11-18T02:56:46.825 INFO:teuthology.results:seed: '4664' 2021-11-18T02:57:43.218 INFO:root:teuthology version: 1.1.0-6cfddd5e 2021-11-18T02:57:43.224 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-11-18T03:58:47.909 INFO:teuthology.results:Tests finished! gathering results... 2021-11-18T03:58:48.544 INFO:scrape:Found 8 jobs 2021-11-18T03:58:50.197 INFO:scrape:Found 2 distinct failure reasons 2021-11-18T03:58:50.198 INFO:scrape:Failure: reached maximum tries (150) after waiting for 900 seconds 2021-11-18T03:58:50.198 INFO:scrape:5 jobs: ['6511190', '6511188', '6511185', '6511186', '6511191'] 2021-11-18T03:58:50.199 INFO:scrape:suites intersection: ['1-volume/{0-create', '2-client', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'roles', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-18T03:58:50.200 INFO:scrape:suites union: ['1-ranks/1', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'roles', 'tasks/{0-from/pacific', 'tasks/{0-from/v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-18T03:58:50.200 INFO:scrape: 2021-11-18T03:58:50.201 INFO:scrape:Failure: Command failed on smithi167 with status 1: 'sudo yum install -y kernel' 2021-11-18T03:58:50.201 INFO:scrape:2 jobs: ['6511189', '6511187'] 2021-11-18T03:58:50.202 INFO:scrape:suites intersection: ['1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'roles', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-18T03:58:50.202 INFO:scrape:suites union: ['1-ranks/1', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'roles', 'tasks/{0-from/pacific', 'tasks/{0-from/v16.2.4', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-18T03:58:50.203 INFO:scrape: 2021-11-18T03:58:50.205 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/ failed: 7 dead: 0 running: 0 waiting: 0 queued: 0 passed: 1 Fail ================================================================= [6511185] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:47:42 info: http://pulpito.ceph.com/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6511185/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6511185/ reached maximum tries (150) after waiting for 900 seconds [6511186] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:45:11 info: http://pulpito.ceph.com/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6511186/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6511186/ reached maximum tries (150) after waiting for 900 seconds [6511187] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6511187/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6511187/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5ca4e6faafd14dab872de1c6e0ad3271 Command failed on smithi016 with status 1: 'sudo yum install -y kernel' [6511188] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:47:18 info: http://pulpito.ceph.com/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6511188/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6511188/ reached maximum tries (150) after waiting for 900 seconds [6511189] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6511189/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6511189/ sentry: https://sentry.ceph.com/organizations/ceph/?query=290f2cebd7e14f6a9dc39f32097c15ce Command failed on smithi167 with status 1: 'sudo yum install -y kernel' [6511190] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:46:55 info: http://pulpito.ceph.com/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6511190/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6511190/ reached maximum tries (150) after waiting for 900 seconds [6511191] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:47:21 info: http://pulpito.ceph.com/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6511191/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6511191/ reached maximum tries (150) after waiting for 900 seconds Pass ================================================================= [6511184] fs:upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:51:16 info: http://pulpito.ceph.com/pdonnell-2021-11-18_02:55:54-fs:upgrade-wip-pdonnell-testing-20211116.201517-distro-basic-smithi/6511184/