2021-01-15T14:24:50.077 INFO:root:teuthology version: 1.0.0-04fddc54 2021-01-15T14:24:50.079 INFO:teuthology.results:seed: '3633' 2021-01-15T14:37:34.119 INFO:root:teuthology version: 1.0.0-04fddc54 2021-01-15T14:37:34.167 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-01-15T19:01:38.686 INFO:teuthology.results:Tests finished! gathering results... 2021-01-15T19:01:41.102 INFO:scrape:Found 16 jobs 2021-01-15T19:01:44.727 INFO:scrape:Found 1 distinct failure reasons 2021-01-15T19:01:44.728 INFO:scrape:Failure: Command failed on smithi105 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' 2021-01-15T19:01:44.729 INFO:scrape:8 jobs: ['5789025', '5789024', '5789017', '5789020', '5789028', '5789021', '5789016', '5789013'] 2021-01-15T19:01:44.729 INFO:scrape:suites intersection: ['1-ceph-install/nautilus', '2-workload/{blogbench', '4-octopus', '5-final-workload/{blogbench', 'ec-rados-default', 'rados-snaps-few-objects', 'rados_api', 'rados_loadgenbig', 'rados_loadgenmix', 'rados_mon_thrash', 'rbd_cls', 'rbd_import_export', 'rgw', 'rgw_ragweed_check}', 'rgw_ragweed_prepare', 'start}', 'test_rbd_api', 'test_rbd_python}', 'ubuntu_latest}', 'upgrade:nautilus-x/parallel/{0-cluster/{openstack'] 2021-01-15T19:01:44.730 INFO:scrape:suites union: ['1-ceph-install/nautilus', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-workload/{blogbench', '3-upgrade-sequence/upgrade-all', '3-upgrade-sequence/upgrade-mon-osd-mds', '4-octopus', '5-final-workload/{blogbench', 'ec-rados-default', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'rados-snaps-few-objects', 'rados_api', 'rados_loadgenbig', 'rados_loadgenmix', 'rados_mon_thrash', 'rbd_cls', 'rbd_import_export', 'rgw', 'rgw_ragweed_check}', 'rgw_ragweed_prepare', 'start}', 'test_rbd_api', 'test_rbd_python}', 'ubuntu_latest}', 'upgrade:nautilus-x/parallel/{0-cluster/{openstack'] 2021-01-15T19:01:44.730 INFO:scrape: 2021-01-15T19:01:44.732 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/ failed: 8 dead: 0 running: 0 waiting: 0 queued: 0 passed: 8 Fail ================================================================= [5789013] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 01:14:14 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789013/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789013/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c58b71ce5beb493d8e139e88bb296faa Command failed on smithi194 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5789016] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 01:10:52 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789016/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789016/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8bc9837a69894562b85c94aa8f528aac Command failed on smithi181 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5789017] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 01:13:51 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789017/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789017/ sentry: https://sentry.ceph.com/organizations/ceph/?query=92a9a7bcd287490f937b1d07f8529910 Command failed on smithi151 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5789020] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 01:12:27 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789020/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789020/ sentry: https://sentry.ceph.com/organizations/ceph/?query=76f9ca0967504a69b7b147803f792b8c Command failed on smithi124 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5789021] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 01:14:24 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789021/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789021/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c695dcfddb364dfd99970b41fcf9234b Command failed on smithi125 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5789024] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 01:10:30 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789024/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789024/ sentry: https://sentry.ceph.com/organizations/ceph/?query=43b1da9cded1449092cf02b167f6df1c Command failed on smithi084 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5789025] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 01:13:50 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789025/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789025/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cbe41f304b3645ff855915b0229c12c1 Command failed on smithi105 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5789028] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 01:11:32 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789028/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789028/ sentry: https://sentry.ceph.com/organizations/ceph/?query=21af55ca1a394c2fa42c79be1d50fbf4 Command failed on smithi175 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' Pass ================================================================= [5789014] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_latest} time: 03:40:53 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789014/ [5789015] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_latest} time: 02:28:52 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789015/ [5789018] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_latest} time: 03:32:14 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789018/ [5789019] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_latest} time: 01:47:31 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789019/ [5789022] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_latest} time: 03:54:00 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789022/ [5789023] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_latest} time: 02:09:42 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789023/ [5789026] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_latest} time: 03:40:29 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789026/ [5789027] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_latest} time: 02:00:49 info: http://pulpito.ceph.com/teuthology-2021-01-15_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5789027/