2020-12-26T02:31:47.365 INFO:root:teuthology version: 1.0.0-8e49dda4 2020-12-26T02:31:47.367 INFO:teuthology.results:seed: '9616' 2020-12-26T02:37:03.161 INFO:root:teuthology version: 1.0.0-8e49dda4 2020-12-26T02:37:03.166 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-12-26T12:55:14.124 INFO:teuthology.results:Tests finished! gathering results... 2020-12-26T12:55:16.480 INFO:scrape:Found 16 jobs 2020-12-26T12:55:23.649 INFO:scrape:Found 1 distinct failure reasons 2020-12-26T12:55:23.650 INFO:scrape:Failure: Command failed on smithi082 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' 2020-12-26T12:55:23.651 INFO:scrape:8 jobs: ['5736958', '5736950', '5736965', '5736961', '5736954', '5736953', '5736962', '5736957'] 2020-12-26T12:55:23.651 INFO:scrape:suites intersection: ['1-ceph-install/mimic', '2-workload/{blogbench', '4-octopus', '5-msgr2', '6-final-workload/{blogbench', 'ec-rados-default', 'rados-snaps-few-objects', 'rados_api', 'rados_loadgenbig', 'rados_loadgenmix', 'rados_mon_thrash', 'rbd_cls', 'rbd_import_export', 'rgw', 'rgw_ragweed_check}', 'rgw_ragweed_prepare', 'start}', 'test_rbd_api', 'test_rbd_python}', 'ubuntu_latest}', 'upgrade:mimic-x/parallel/{0-cluster/{openstack'] 2020-12-26T12:55:23.652 INFO:scrape:suites union: ['1-ceph-install/mimic', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-workload/{blogbench', '3-upgrade-sequence/upgrade-all', '3-upgrade-sequence/upgrade-mon-osd-mds', '4-octopus', '5-msgr2', '6-final-workload/{blogbench', 'ec-rados-default', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'rados-snaps-few-objects', 'rados_api', 'rados_loadgenbig', 'rados_loadgenmix', 'rados_mon_thrash', 'rbd_cls', 'rbd_import_export', 'rgw', 'rgw_ragweed_check}', 'rgw_ragweed_prepare', 'start}', 'test_rbd_api', 'test_rbd_python}', 'ubuntu_latest}', 'upgrade:mimic-x/parallel/{0-cluster/{openstack'] 2020-12-26T12:55:23.652 INFO:scrape: 2020-12-26T12:55:23.654 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/ failed: 8 dead: 0 running: 0 waiting: 0 queued: 0 passed: 8 Fail ================================================================= [5736950] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 01:12:27 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736950/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736950/ sentry: https://sentry.ceph.com/organizations/ceph/?query=79385d756ceb4ea9857570f248751cc3 Command failed on smithi148 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5736953] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 01:08:01 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736953/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736953/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e671e1b52be1475d9fa62f5a4e5d0b30 Command failed on smithi106 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5736954] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 01:11:26 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736954/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736954/ sentry: https://sentry.ceph.com/organizations/ceph/?query=50e4295747ea4906a1d56799e2393232 Command failed on smithi085 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5736957] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 01:09:34 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736957/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736957/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0653470897c8417aa951c03cb47460d6 Command failed on smithi087 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5736958] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 01:11:03 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736958/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736958/ sentry: https://sentry.ceph.com/organizations/ceph/?query=66e98b7c53a44b20a725eb70e3f49fc8 Command failed on smithi082 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5736961] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 01:08:50 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736961/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736961/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5d1d2d4d90bc444488fd3a221bdbd485 Command failed on smithi188 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5736962] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 01:13:07 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736962/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736962/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4464b8a3d25d4beabc3e353e89ccc118 Command failed on smithi033 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5736965] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 01:09:15 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736965/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736965/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ee867683c1094cbcbd0616eb54a6da62 Command failed on smithi062 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' Pass ================================================================= [5736951] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_latest} time: 05:14:59 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736951/ [5736952] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/bluestore-bitmap thrashosds-health ubuntu_latest} time: 00:50:37 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736952/ [5736955] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_latest} time: 03:12:49 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736955/ [5736956] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/filestore-xfs thrashosds-health ubuntu_latest} time: 00:48:03 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736956/ [5736959] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_latest} time: 05:05:57 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736959/ [5736960] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/bluestore-bitmap thrashosds-health ubuntu_latest} time: 00:48:09 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736960/ [5736963] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_latest} time: 03:17:17 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736963/ [5736964] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/filestore-xfs thrashosds-health ubuntu_latest} time: 00:51:00 info: http://pulpito.ceph.com/teuthology-2020-12-26_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5736964/