2022-06-25T01:52:35.746 INFO:root:teuthology version: 1.1.1.dev376+g082ae7ef 2022-06-25T01:52:35.749 INFO:teuthology.results:seed: '639' 2022-06-25T02:08:24.725 INFO:root:teuthology version: 1.1.1.dev376+g082ae7ef 2022-06-25T02:08:24.729 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-06-25T05:48:39.913 INFO:teuthology.results:Tests finished! gathering results... 2022-06-25T05:48:41.628 INFO:scrape:Found 16 jobs 2022-06-25T05:48:41.630 INFO:scrape:Found 1 distinct failure reasons 2022-06-25T05:48:41.631 INFO:scrape:Timeout 120 running ceph --cluster ceph osd dump --format=json 2022-06-25T05:48:41.631 INFO:scrape:1 jobs: ['6897433'] 2022-06-25T05:48:41.632 INFO:scrape:suites: ['1-ceph-install/nautilus', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{radosbench', '5-finish-upgrade', '6-octopus', '7-msgr2', '8-final-workload/{rbd-python', 'objectstore/filestore-xfs', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_18.04}', 'upgrade:nautilus-x/stress-split/{0-cluster/{openstack'] 2022-06-25T05:48:41.632 INFO:scrape: 2022-06-25T05:48:41.633 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 15 Fail ================================================================= [6897433] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 01:18:25 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897433/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897433/ sentry: https://sentry.ceph.com/organizations/ceph/?query=41c14df9270c462688315ec98225139d Command failed on smithi022 with status 124: 'sudo adjust-ulimits ceph- coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' Pass ================================================================= [6897428] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_18.04} time: 02:07:42 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897428/ [6897429] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_18.04} time: 03:20:11 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897429/ [6897430] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_18.04} time: 01:47:55 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897430/ [6897431] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_18.04} time: 02:02:28 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897431/ [6897432] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_18.04} time: 02:11:29 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897432/ [6897434] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_18.04} time: 01:16:34 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897434/ [6897435] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_18.04} time: 02:03:08 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897435/ [6897436] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_18.04} time: 02:06:50 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897436/ [6897437] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_18.04} time: 03:36:10 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897437/ [6897438] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_18.04} time: 01:35:37 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897438/ [6897439] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_18.04} time: 02:03:29 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897439/ [6897440] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_18.04} time: 02:07:55 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897440/ [6897441] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_18.04} time: 02:47:23 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897441/ [6897442] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_18.04} time: 01:29:36 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897442/ [6897443] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_18.04} time: 02:02:54 info: http://pulpito.ceph.com/teuthology-2022-06-24_14:23:03-upgrade:nautilus-x-octopus-distro-default-smithi/6897443/