2022-02-21T03:16:27.894 INFO:root:teuthology version: 1.1.0-eea25212 2022-02-21T03:16:27.895 INFO:teuthology.results:subset: '5/10' 2022-02-21T03:16:27.895 INFO:teuthology.results:seed: '4845' 2022-02-21T06:52:17.256 INFO:root:teuthology version: 1.1.0-eea25212 2022-02-21T06:52:17.288 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-02-21T11:28:37.449 INFO:teuthology.results:Tests finished! gathering results... 2022-02-21T11:28:41.990 INFO:scrape:Found 34 jobs 2022-02-21T11:28:42.141 INFO:scrape:Found 1 distinct failure reasons 2022-02-21T11:28:42.142 INFO:scrape:Failure: SSH connection to smithi181 was lost: 'uname -r' 2022-02-21T11:28:42.143 INFO:scrape:3 jobs: ['6696178', '6696186', '6696181'] 2022-02-21T11:28:42.143 INFO:scrape:suites intersection: ['rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}'] 2022-02-21T11:28:42.144 INFO:scrape:suites union: ['0-roles', '1-ceph-install/octopus', '1-start', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-first-half-tasks/rbd-import-export', '2-partial-upgrade/firsthalf', '3-stress-tasks/{radosbench', '3-thrash/default', '4-second-half-tasks/rbd-import-export', '4-workload/{radosbench', '5-finish-upgrade', '6-pacific', '8-final-workload/{rbd-python', 'mon_election/classic', 'mon_election/classic}', 'mon_election/connectivity', 'objectstore/bluestore-hybrid', 'objectstore/filestore-xfs', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_18.04}', 'upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack', 'upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04'] 2022-02-21T11:28:42.144 INFO:scrape: 2022-02-21T11:28:42.147 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/ failed: 0 dead: 3 running: 0 waiting: 0 queued: 0 passed: 31 Dead ================================================================= [6696178] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/bluestore-hybrid thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696178/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696178/ sentry: https://sentry.ceph.com/organizations/ceph/?query=20f8105555fa4123b8c5567f53985705 SSH connection to smithi181 was lost: 'uname -r' [6696181] upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696181/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696181/ sentry: https://sentry.ceph.com/organizations/ceph/?query=00e162ce218645d9b1404185e0ead8ac SSH connection to smithi183 was lost: 'uname -r' [6696186] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/filestore-xfs thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696186/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696186/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a79ebaa3a77048568fe62e97e6b92eda SSH connection to smithi080 was lost: 'uname -r' Pass ================================================================= [6696157] upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:02:42 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696157/ [6696158] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-all 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} time: 00:21:08 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696158/ [6696159] upgrade:octopus-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 01:54:29 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696159/ [6696160] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-hybrid 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04} time: 04:13:29 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696160/ [6696161] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/bluestore-hybrid thrashosds-health ubuntu_18.04} time: 05:16:22 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696161/ [6696162] upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:05:10 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696162/ [6696163] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_3.0 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:16:50 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696163/ [6696164] upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 01:50:12 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696164/ [6696165] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_rhel8 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:19:32 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696165/ [6696166] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-mon-osd-mds 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} time: 00:27:18 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696166/ [6696167] upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:06:02 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696167/ [6696168] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04} time: 02:01:05 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696168/ [6696169] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/filestore-xfs thrashosds-health ubuntu_18.04} time: 02:43:39 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696169/ [6696170] upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:05:45 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696170/ [6696171] upgrade:octopus-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 01:59:29 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696171/ [6696172] upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:08:37 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696172/ [6696173] upgrade:octopus-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:05:53 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696173/ [6696174] upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:02:52 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696174/ [6696175] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-all 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} time: 00:20:55 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696175/ [6696176] upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:18:12 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696176/ [6696177] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04} time: 01:25:56 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696177/ [6696179] upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:22:37 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696179/ [6696180] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_3.0 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:18:48 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696180/ [6696182] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_rhel8 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:17:24 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696182/ [6696183] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-mon-osd-mds 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} time: 00:27:46 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696183/ [6696184] upgrade:octopus-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:29:09 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696184/ [6696185] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-hybrid 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04} time: 04:32:11 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696185/ [6696187] upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:06:03 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696187/ [6696188] upgrade:octopus-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 01:42:19 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696188/ [6696189] upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:06:51 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696189/ [6696190] upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:20:37 info: http://pulpito.ceph.com/teuthology-2022-02-19_14:22:03-upgrade:octopus-x-pacific-distro-default-smithi/6696190/