2021-11-03T20:14:43.095 INFO:root:teuthology version: 1.1.0-c56135d1 2021-11-03T20:14:43.097 INFO:teuthology.results:subset: '111/120000' 2021-11-03T20:14:43.097 INFO:teuthology.results:seed: '4501' 2021-11-03T20:53:31.242 INFO:root:teuthology version: 1.1.0-c56135d1 2021-11-03T20:53:31.278 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-11-04T02:12:55.657 INFO:teuthology.results:Tests finished! gathering results... 2021-11-04T02:12:58.818 INFO:scrape:Found 31 jobs 2021-11-04T02:12:58.925 INFO:scrape:Found 1 distinct failure reasons 2021-11-04T02:12:58.926 INFO:scrape:Failure: Could not reconnect to ubuntu@smithi040.front.sepia.ceph.com 2021-11-04T02:12:58.926 INFO:scrape:2 jobs: ['6483271', '6483275'] 2021-11-04T02:12:58.927 INFO:scrape:suites intersection: [] 2021-11-04T02:12:58.927 INFO:scrape:suites union: ['0-start', '1-ceph-install/octopus', '1-tasks', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{radosbench', '5-finish-upgrade', '6-pacific', '7-msgr2', '8-final-workload/{rbd-python', 'mon_election/classic', 'mon_election/connectivity', 'objectstore/filestore-xfs', 'rados_api', 'rados_loadgenbig', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'rbd_import_export', 'readwrite', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'test_rbd_api}}', 'thrashosds-health', 'ubuntu_18.04}', 'upgrade-sequence', 'upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04', 'upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack', 'workload/{ec-rados-default'] 2021-11-04T02:12:58.927 INFO:scrape: 2021-11-04T02:12:58.930 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/ failed: 2 dead: 0 running: 0 waiting: 0 queued: 0 passed: 29 Fail ================================================================= [6483271] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/filestore-xfs thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483271/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483271/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ee2a638e0eb347d8bcce2a495aa2b11a Could not reconnect to ubuntu@smithi040.front.sepia.ceph.com [6483275] upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483275/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483275/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a4be3739a46548b1bdfde52e26e8b27c Could not reconnect to ubuntu@smithi039.front.sepia.ceph.com Pass ================================================================= [6483266] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-mon-osd-mds 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} time: 00:26:54 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483266/ [6483269] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-nautilus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04} time: 00:47:42 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483269/ [6483273] upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:02:46 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483273/ [6483277] upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:09:29 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483277/ [6483279] upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:14:32 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483279/ [6483281] upgrade:octopus-x/parallel/{0-distro/centos_8.2_container_tools_3.0 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:05:56 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483281/ [6483283] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-all 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} time: 00:20:48 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483283/ [6483285] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-nautilus-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04} time: 00:47:44 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483285/ [6483287] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/bluestore-bitmap thrashosds-health ubuntu_18.04} time: 04:40:51 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483287/ [6483289] upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:11:37 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483289/ [6483291] upgrade:octopus-x/parallel/{0-distro/centos_8.3_container_tools_3.0 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:11:45 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483291/ [6483293] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_3.0 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:16:12 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483293/ [6483295] upgrade:octopus-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:19:04 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483295/ [6483297] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_rhel8 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:16:29 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483297/ [6483299] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-mon-osd-mds 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} time: 00:26:40 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483299/ [6483301] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-nautilus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04} time: 00:49:51 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483301/ [6483303] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/filestore-xfs thrashosds-health ubuntu_18.04} time: 02:37:55 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483303/ [6483305] upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:05:55 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483305/ [6483307] upgrade:octopus-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:28:14 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483307/ [6483308] upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:09:06 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483308/ [6483309] upgrade:octopus-x/stress-split/{0-distro/centos_8.2_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 01:55:55 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483309/ [6483310] upgrade:octopus-x/parallel/{0-distro/centos_8.2_container_tools_3.0 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:06:39 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483310/ [6483311] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-all 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} time: 00:20:08 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483311/ [6483312] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-nautilus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04} time: 00:49:45 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483312/ [6483313] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/bluestore-bitmap thrashosds-health ubuntu_18.04} time: 05:12:33 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483313/ [6483314] upgrade:octopus-x/parallel/{0-distro/centos_8.3_container_tools_3.0 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:10:23 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483314/ [6483315] upgrade:octopus-x/stress-split/{0-distro/centos_8.3_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:03:12 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483315/ [6483316] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_3.0 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:17:23 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483316/ [6483317] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_rhel8 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:16:11 info: http://pulpito.ceph.com/yuriw-2021-11-03_20:12:49-upgrade:octopus-x-wip-yuri4-testing-2021-11-03-1142-pacific-distro-basic-smithi/6483317/