2022-04-28T23:33:00.661 INFO:root:teuthology version: 1.1.1.dev328+g004701de 2022-04-28T23:33:00.661 INFO:teuthology.results:subset: '111/120000' 2022-04-28T23:33:00.661 INFO:teuthology.results:seed: '9552' 2022-04-29T00:10:07.825 INFO:root:teuthology version: 1.1.1.dev328+g004701de 2022-04-29T00:10:07.845 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-04-29T05:03:30.554 INFO:teuthology.results:Tests finished! gathering results... 2022-04-29T05:03:33.557 INFO:scrape:Found 29 jobs 2022-04-29T05:03:33.568 INFO:scrape:Found 1 distinct failure reasons 2022-04-29T05:03:33.568 INFO:scrape:Failure: Command failed on smithi059 with status 123: "sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --" 2022-04-29T05:03:33.569 INFO:scrape:1 jobs: ['6811976'] 2022-04-29T05:03:33.569 INFO:scrape:suites: ['1-ceph-install/octopus', '1.1-pg-log-overrides/normal_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{radosbench', '5-finish-upgrade', '6-pacific', '8-final-workload/{rbd-python', 'mon_election/connectivity', 'objectstore/filestore-xfs', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_18.04}', 'upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack'] 2022-04-29T05:03:33.569 INFO:scrape: 2022-04-29T05:03:33.571 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 28 Fail ================================================================= [6811976] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/filestore-xfs thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 02:28:41 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811976/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811976/ Command failed on smithi059 with status 123: "sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --" Pass ================================================================= [6811959] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-mon-osd-mds 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} time: 00:26:52 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811959/ [6811960] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-hybrid 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04} time: 04:25:54 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811960/ [6811961] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/filestore-xfs thrashosds-health ubuntu_18.04} time: 02:37:12 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811961/ [6811962] upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 01:50:51 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811962/ [6811963] upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:07:06 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811963/ [6811964] upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:09:47 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811964/ [6811965] upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:10:05 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811965/ [6811966] upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:03:41 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811966/ [6811967] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-all 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} time: 00:21:01 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811967/ [6811968] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-hybrid 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04} time: 04:03:36 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811968/ [6811969] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/bluestore-hybrid thrashosds-health ubuntu_18.04} time: 05:10:28 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811969/ [6811970] upgrade:octopus-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:02:34 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811970/ [6811971] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_3.0 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:16:09 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811971/ [6811972] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_rhel8 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:13:56 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811972/ [6811973] upgrade:octopus-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:09:27 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811973/ [6811974] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-mon-osd-mds 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} time: 00:27:47 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811974/ [6811975] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04} time: 01:50:49 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811975/ [6811977] upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:06:53 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811977/ [6811978] upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:14:15 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811978/ [6811979] upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:09:15 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811979/ [6811980] upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:12:09 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811980/ [6811981] upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:04:49 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811981/ [6811982] upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-workload/{rgw_ragweed_prepare} 3-upgrade-sequence/upgrade-all 4-pacific 5-final-workload/{rgw rgw_ragweed_check} centos_latest} time: 00:21:43 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811982/ [6811983] upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-pacific 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04} time: 01:27:31 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811983/ [6811984] upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-pacific 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/bluestore-hybrid thrashosds-health ubuntu_18.04} time: 04:47:44 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811984/ [6811985] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_3.0 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:17:47 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811985/ [6811986] upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 01:55:07 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811986/ [6811987] upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_rhel8 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:14:06 info: http://pulpito.ceph.com/yuriw-2022-04-28_23:25:04-upgrade:octopus-x-wip-55444-pacific-distro-default-smithi/6811987/