2021-10-16T14:22:32.305 INFO:root:teuthology version: 1.1.0-b35344e8
2021-10-16T14:22:32.306 INFO:teuthology.results:subset: '2/10'
2021-10-16T14:22:32.306 INFO:teuthology.results:seed: '5490'
2021-10-16T14:22:54.360 INFO:root:teuthology version: 1.1.0-b35344e8
2021-10-16T14:22:54.365 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-10-16T19:13:16.399 INFO:teuthology.results:Tests finished! gathering results...
2021-10-16T19:13:18.960 INFO:scrape:Found 25 jobs
2021-10-16T19:13:19.204 INFO:scrape:Found 1 distinct failure reasons
2021-10-16T19:13:19.205 INFO:scrape:Timeout 120 running ceph --cluster ceph pg dump --format=json
2021-10-16T19:13:19.206 INFO:scrape:4 jobs: ['6445975', '6445976', '6445971', '6445970']
2021-10-16T19:13:19.206 INFO:scrape:suites intersection: ['start}']
2021-10-16T19:13:19.207 INFO:scrape:suites union: ['1-ceph-install/octopus', '1-nautilus-install/octopus', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '2-workload/{rgw_ragweed_prepare}', '3-thrash/default', '3-upgrade-sequence/upgrade-mon-osd-mds', '3.1-objectstore/bluestore-bitmap', '3.1-objectstore/filestore-xfs', '4-ec-workload/{rados-ec-workload', '4-pacific', '4-workload/{radosbench', '5-final-workload/{rgw', '5-finish-upgrade', '6-pacific', '7-final-workload', '7-msgr2', '8-final-workload/{rbd-python', 'centos_latest}', 'mon_election/classic', 'mon_election/connectivity', 'objectstore/bluestore-bitmap', 'rbd-cls', 'rbd-ec-workload}', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw_ragweed_check}', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_18.04}', 'upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack', 'upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack', 'upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack']
2021-10-16T19:13:19.207 INFO:scrape:
2021-10-16T19:13:19.210 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/
failed:  4
dead:    0
running: 0
waiting: 0
queued:  0
passed:  21


Fail
=================================================================
[6445970]  upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-nautilus-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04}
-----------------------------------------------------------------
time:   00:26:16
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445970/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445970/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b2f00e43815b4ed7ae478ade6224f13e

    Command failed on smithi135 with status 124: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph pg dump --format=json'

[6445971]  upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/bluestore-bitmap thrashosds-health ubuntu_18.04}
-----------------------------------------------------------------
time:   00:23:18
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445971/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445971/
sentry: https://sentry.ceph.com/organizations/ceph/?query=71a7119206b14d818a96be6508d9221d

    Command failed on smithi137 with status 124: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph pg dump --format=json'

[6445975]  upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-mon-osd-mds 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest}
-----------------------------------------------------------------
time:   00:24:02
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445975/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445975/
sentry: https://sentry.ceph.com/organizations/ceph/?query=eb2260c41b034ac4bf1bc1302a0529c4

    Command failed on smithi099 with status 124: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph pg dump --format=json'

[6445976]  upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-nautilus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04}
-----------------------------------------------------------------
time:   00:26:34
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445976/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445976/
sentry: https://sentry.ceph.com/organizations/ceph/?query=16721f11a2e746a792290548e8c31225

    Command failed on smithi155 with status 124: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph pg dump --format=json'



Pass
=================================================================
[6445967] upgrade:octopus-x/stress-split/{0-distro/centos_8.3_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic}
time:   02:12:08
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445967/

[6445968] upgrade:octopus-x/parallel/{0-distro/centos_8.3_container_tools_3.0 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}}
time:   01:07:15
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445968/

[6445969] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-all 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest}
time:   00:22:25
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445969/

[6445972] upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity}
time:   01:58:22
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445972/

[6445973] upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}}
time:   01:07:24
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445973/

[6445974] upgrade:octopus-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic}
time:   02:11:04
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445974/

[6445977] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/filestore-xfs thrashosds-health ubuntu_18.04}
time:   02:43:08
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445977/

[6445978] upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}}
time:   01:09:10
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445978/

[6445979] upgrade:octopus-x/stress-split/{0-distro/centos_8.3_container_tools_3.0 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity}
time:   02:05:01
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445979/

[6445980] upgrade:octopus-x/parallel/{0-distro/centos_8.3_container_tools_3.0 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}}
time:   01:06:52
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445980/

[6445981] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-all 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest}
time:   00:20:52
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445981/

[6445982] upgrade:octopus-x/stress-split/{0-distro/centos_8.3_container_tools_3.0 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity}
time:   02:18:18
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445982/

[6445983] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-nautilus-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04}
time:   00:50:45
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445983/

[6445984] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/bluestore-bitmap thrashosds-health ubuntu_18.04}
time:   04:41:05
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445984/

[6445985] upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic}
time:   01:57:10
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445985/

[6445986] upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}}
time:   01:07:04
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445986/

[6445987] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-mon-osd-mds 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest}
time:   00:27:29
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445987/

[6445988] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-nautilus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04}
time:   00:54:10
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445988/

[6445989] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/filestore-xfs thrashosds-health ubuntu_18.04}
time:   02:55:47
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445989/

[6445990] upgrade:octopus-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity}
time:   02:15:05
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445990/

[6445991] upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}}
time:   01:10:19
info:   http://pulpito.ceph.com/teuthology-2021-10-16_14:22:02-upgrade:octopus-x-pacific-distro-default-smithi/6445991/