2022-04-28T16:54:34.059 INFO:root:teuthology version: 1.1.1.dev328+g004701de 2022-04-28T16:54:34.059 INFO:teuthology.results:subset: '111/120000' 2022-04-28T16:54:34.060 INFO:teuthology.results:seed: '8785' 2022-04-28T20:04:38.569 INFO:root:teuthology version: 1.1.1.dev328+g004701de 2022-04-28T20:04:38.733 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-04-29T02:02:06.065 INFO:teuthology.results:Tests finished! gathering results... 2022-04-29T02:02:08.851 INFO:scrape:Found 27 jobs 2022-04-29T02:02:19.920 INFO:scrape:Found 6 distinct failure reasons 2022-04-29T02:02:19.921 INFO:scrape:Failure: Command failed (workunit test cls/test_cls_journal.sh) on smithi036 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_journal.sh' 2022-04-29T02:02:19.921 INFO:scrape:10 jobs: ['6810850', '6810843', '6810842', '6810852', '6810845', '6810863', '6810861', '6810859', '6810869', '6810867'] 2022-04-29T02:02:19.921 INFO:scrape:suites intersection: ['0-start', '1-tasks', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api}}', 'upgrade-sequence', 'workload/{ec-rados-default'] 2022-04-29T02:02:19.921 INFO:scrape:suites union: ['0-start', '1-tasks', 'mon_election/classic', 'mon_election/connectivity', 'rados_api', 'rados_loadgenbig', 'rbd_import_export', 'test_rbd_api}}', 'upgrade-sequence', 'upgrade:octopus-x/parallel/{0-distro/centos_8.stream_container_tools', 'upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_3.0', 'upgrade:octopus-x/parallel/{0-distro/rhel_8.4_container_tools_rhel8', 'upgrade:octopus-x/parallel/{0-distro/ubuntu_18.04', 'upgrade:octopus-x/parallel/{0-distro/ubuntu_20.04', 'workload/{ec-rados-default'] 2022-04-29T02:02:19.921 INFO:scrape: 2022-04-29T02:02:19.922 INFO:scrape:Timeout 120 running ceph --cluster ceph osd dump --format=json 2022-04-29T02:02:19.922 INFO:scrape:2 jobs: ['6810866', '6810848'] 2022-04-29T02:02:19.922 INFO:scrape:suites intersection: ['1-ceph-install/octopus', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{radosbench', '5-finish-upgrade', '6-pacific', '8-final-workload/{rbd-python', 'objectstore/bluestore-hybrid', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_18.04}', 'upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack'] 2022-04-29T02:02:19.922 INFO:scrape:suites union: ['1-ceph-install/octopus', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{radosbench', '5-finish-upgrade', '6-pacific', '8-final-workload/{rbd-python', 'mon_election/classic', 'mon_election/connectivity', 'objectstore/bluestore-hybrid', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_18.04}', 'upgrade:octopus-x/stress-split-no-cephadm/{0-cluster/{openstack'] 2022-04-29T02:02:19.922 INFO:scrape: 2022-04-29T02:02:19.922 INFO:scrape:Failure: Command failed (workunit test cls/test_cls_rbd.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh' 2022-04-29T02:02:19.923 INFO:scrape:6 jobs: ['6810860', '6810868', '6810862', '6810854', '6810849', '6810841'] 2022-04-29T02:02:19.923 INFO:scrape:suites intersection: ['0-roles', '1-start', '3-stress-tasks/{radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}'] 2022-04-29T02:02:19.923 INFO:scrape:suites union: ['0-roles', '1-start', '2-first-half-tasks/radosbench', '2-first-half-tasks/rbd-cls', '2-first-half-tasks/rbd-import-export', '2-first-half-tasks/readwrite', '2-first-half-tasks/snaps-few-objects', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', '4-second-half-tasks/rbd-import-export', 'mon_election/classic}', 'mon_election/connectivity}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:octopus-x/stress-split/{0-distro/centos_8.stream_container_tools', 'upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0', 'upgrade:octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8', 'upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04', 'upgrade:octopus-x/stress-split/{0-distro/ubuntu_20.04'] 2022-04-29T02:02:19.923 INFO:scrape: 2022-04-29T02:02:19.923 INFO:scrape:Dead: b'2022-04-29T01:10:24.673 DEBUG:teuthology.exit:Finished running handlers' 2022-04-29T02:02:19.923 INFO:scrape:4 jobs: ['6810847', '6810840', '6810865', '6810858'] 2022-04-29T02:02:19.923 INFO:scrape:suites intersection: ['1-octopus-install/octopus', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-ec-workload/{rados-ec-workload', '5-finish-upgrade', '6-pacific', '7-final-workload', 'rbd-ec-workload}', 'start}', 'thrashosds-health', 'ubuntu_18.04}', 'upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack'] 2022-04-29T02:02:19.924 INFO:scrape:suites union: ['1-octopus-install/octopus', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '3.1-objectstore/bluestore-hybrid', '3.1-objectstore/filestore-xfs', '4-ec-workload/{rados-ec-workload', '5-finish-upgrade', '6-pacific', '7-final-workload', 'mon_election/classic', 'mon_election/connectivity', 'rbd-ec-workload}', 'start}', 'thrashosds-health', 'ubuntu_18.04}', 'upgrade:octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack'] 2022-04-29T02:02:19.924 INFO:scrape: 2022-04-29T02:02:19.924 INFO:scrape:Failure: Command failed (ragweed tests against rgw) on smithi124 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.1.conf RAGWEED_STAGES=check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" 2022-04-29T02:02:19.924 INFO:scrape:4 jobs: ['6810839', '6810856', '6810846', '6810864'] 2022-04-29T02:02:19.924 INFO:scrape:suites intersection: ['1-ceph-install/octopus', '2-workload/{rgw_ragweed_prepare}', '4-pacific', '5-final-workload/{rgw', 'centos_latest}', 'rgw_ragweed_check}', 'start}', 'upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack'] 2022-04-29T02:02:19.924 INFO:scrape:suites union: ['1-ceph-install/octopus', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-workload/{rgw_ragweed_prepare}', '3-upgrade-sequence/upgrade-all', '3-upgrade-sequence/upgrade-mon-osd-mds', '4-pacific', '5-final-workload/{rgw', 'centos_latest}', 'rgw_ragweed_check}', 'start}', 'upgrade:octopus-x/parallel-no-cephadm/{0-cluster/{openstack'] 2022-04-29T02:02:19.924 INFO:scrape: 2022-04-29T02:02:19.925 INFO:scrape:Failure: Command failed (workunit test rbd/test_librbd.sh) on smithi064 with status 139: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=octopus TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' 2022-04-29T02:02:19.925 INFO:scrape:1 jobs: ['6810844'] 2022-04-29T02:02:19.925 INFO:scrape:suites: ['0-roles', '1-start', '2-first-half-tasks/rbd_api', '3-stress-tasks/{radosbench', '4-second-half-tasks/radosbench', 'mon_election/connectivity}', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'snaps-few-objects}', 'upgrade:octopus-x/stress-split/{0-distro/ubuntu_18.04'] 2022-04-29T02:02:19.925 INFO:scrape: