2021-02-06T02:31:06.638 INFO:root:teuthology version: 1.1.0-17c09965 2021-02-06T02:31:06.640 INFO:teuthology.results:seed: '5160' 2021-02-06T03:20:03.009 INFO:root:teuthology version: 1.1.0-17c09965 2021-02-06T03:20:03.110 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-02-06T09:06:09.224 INFO:teuthology.results:Tests finished! gathering results... 2021-02-06T09:06:12.234 INFO:scrape:Found 16 jobs 2021-02-06T09:06:33.894 INFO:scrape:Found 1 distinct failure reasons 2021-02-06T09:06:33.895 INFO:scrape:Failure: Command failed on smithi195 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' 2021-02-06T09:06:33.896 INFO:scrape:8 jobs: ['5860214', '5860221', '5860222', '5860218', '5860217', '5860225', '5860226', '5860229'] 2021-02-06T09:06:33.896 INFO:scrape:suites intersection: ['1-ceph-install/mimic', '2-workload/{blogbench', '4-octopus', '5-msgr2', '6-final-workload/{blogbench', 'ec-rados-default', 'rados-snaps-few-objects', 'rados_api', 'rados_loadgenbig', 'rados_loadgenmix', 'rados_mon_thrash', 'rbd_cls', 'rbd_import_export', 'rgw', 'rgw_ragweed_check}', 'rgw_ragweed_prepare', 'start}', 'test_rbd_api', 'test_rbd_python}', 'ubuntu_latest}', 'upgrade:mimic-x/parallel/{0-cluster/{openstack'] 2021-02-06T09:06:33.897 INFO:scrape:suites union: ['1-ceph-install/mimic', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-workload/{blogbench', '3-upgrade-sequence/upgrade-all', '3-upgrade-sequence/upgrade-mon-osd-mds', '4-octopus', '5-msgr2', '6-final-workload/{blogbench', 'ec-rados-default', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'rados-snaps-few-objects', 'rados_api', 'rados_loadgenbig', 'rados_loadgenmix', 'rados_mon_thrash', 'rbd_cls', 'rbd_import_export', 'rgw', 'rgw_ragweed_check}', 'rgw_ragweed_prepare', 'start}', 'test_rbd_api', 'test_rbd_python}', 'ubuntu_latest}', 'upgrade:mimic-x/parallel/{0-cluster/{openstack'] 2021-02-06T09:06:33.897 INFO:scrape: 2021-02-06T09:06:33.899 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/ failed: 8 dead: 0 running: 0 waiting: 0 queued: 0 passed: 8 Fail ================================================================= [5860214] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 04:51:00 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860214/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860214/ sentry: https://sentry.ceph.com/organizations/ceph/?query=16a4765c0faa487ca2e779f3e7e7ba49 Command failed on smithi195 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5860217] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 04:44:22 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860217/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860217/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9af09319f0174b288649081035e3fba5 Command failed on smithi185 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5860218] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 04:56:30 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860218/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860218/ sentry: https://sentry.ceph.com/organizations/ceph/?query=366ccfadd6b9490688398eb81d07271d Command failed on smithi190 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5860221] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 04:48:22 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860221/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860221/ sentry: https://sentry.ceph.com/organizations/ceph/?query=00e430717470469b81fbdfc801b0572e Command failed on smithi072 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5860222] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 04:55:29 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860222/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860222/ sentry: https://sentry.ceph.com/organizations/ceph/?query=175499015925470e9c9516d09b32e75e Command failed on smithi189 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5860225] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 04:41:11 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860225/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860225/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3cd52d8fa1ef4637a551669a0c691fb6 Command failed on smithi095 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5860226] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 04:52:00 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860226/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860226/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ad57151aa18046ff8aeedad27ca3aae3 Command failed on smithi140 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5860229] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 04:51:25 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860229/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860229/ sentry: https://sentry.ceph.com/organizations/ceph/?query=36c42a464f6842c0b081542b12a25329 Command failed on smithi172 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' Pass ================================================================= [5860215] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_latest} time: 05:18:03 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860215/ [5860216] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/bluestore-bitmap thrashosds-health ubuntu_latest} time: 00:55:03 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860216/ [5860219] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_latest} time: 03:48:09 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860219/ [5860220] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/filestore-xfs thrashosds-health ubuntu_latest} time: 00:50:01 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860220/ [5860223] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_latest} time: 05:38:11 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860223/ [5860224] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/bluestore-bitmap thrashosds-health ubuntu_latest} time: 00:51:39 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860224/ [5860227] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_latest} time: 03:42:14 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860227/ [5860228] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/filestore-xfs thrashosds-health ubuntu_latest} time: 00:48:59 info: http://pulpito.ceph.com/teuthology-2021-02-06_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5860228/