2021-06-24T01:25:30.007 INFO:root:teuthology version: 1.1.0-26631ede 2021-06-24T01:25:30.008 INFO:teuthology.results:seed: '5161' 2021-06-24T01:25:43.975 INFO:root:teuthology version: 1.1.0-26631ede 2021-06-24T01:25:43.980 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-06-24T05:34:01.990 INFO:teuthology.results:Tests finished! gathering results... 2021-06-24T05:34:04.062 INFO:scrape:Found 16 jobs 2021-06-24T05:34:04.064 INFO:scrape:Found 1 distinct failure reasons 2021-06-24T05:34:04.064 INFO:scrape:Failure: "2021-06-24 03:36:18.049404 osd.6 (osd.6) 103 : cluster [ERR] 2.1a deep-scrub : stat mismatch, got 1/1 objects, 0/0 clones, 1/1 dirty, 0/0 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 19/0 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes." in cluster log 2021-06-24T05:34:04.065 INFO:scrape:1 jobs: ['6187668'] 2021-06-24T05:34:04.065 INFO:scrape:suites: ['1-ceph-install/nautilus', '1.1.short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{fsx', '5-finish-upgrade', '7-final-workload/{rbd-python', 'objectstore/default', 'radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw-swift', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'supported-all-distro/centos_latest', 'thrashosds-health}', 'upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack'] 2021-06-24T05:34:04.066 INFO:scrape: 2021-06-24T05:34:04.067 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 15 Fail ================================================================= [6187668] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/centos_latest thrashosds-health} ----------------------------------------------------------------- time: 02:49:05 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187668/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187668/ "2021-06-24 03:36:18.049404 osd.6 (osd.6) 103 : cluster [ERR] 2.1a deep- scrub : stat mismatch, got 1/1 objects, 0/0 clones, 1/1 dirty, 0/0 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 19/0 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes." in cluster log Pass ================================================================= [6187662] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/centos_latest} time: 01:10:13 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187662/ [6187663] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/centos_latest thrashosds-health} time: 03:41:50 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187663/ [6187664] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/rhel_7 thrashosds-health} time: 03:18:23 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187664/ [6187665] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/ubuntu_16.04 thrashosds-health} time: 02:44:06 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187665/ [6187666] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/rhel_7} time: 01:10:38 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187666/ [6187667] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest thrashosds-health} time: 03:58:05 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187667/ [6187669] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/rhel_7 thrashosds-health} time: 02:37:10 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187669/ [6187670] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/ubuntu_16.04} time: 01:07:10 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187670/ [6187671] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_16.04 thrashosds-health} time: 03:21:07 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187671/ [6187672] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/ubuntu_latest thrashosds-health} time: 02:57:51 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187672/ [6187673] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/centos_latest thrashosds-health} time: 02:40:57 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187673/ [6187674] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/ubuntu_latest} time: 01:05:03 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187674/ [6187675] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/rhel_7 thrashosds-health} time: 03:31:29 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187675/ [6187676] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/ubuntu_16.04 thrashosds-health} time: 02:42:09 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187676/ [6187677] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/ubuntu_latest thrashosds-health} time: 03:00:41 info: http://pulpito.ceph.com/teuthology-2021-06-24_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6187677/