2021-05-20T12:30:24.552 INFO:root:teuthology version: 1.1.0-471a2f1e 2021-05-20T12:30:24.554 INFO:teuthology.results:seed: '700' 2021-05-20T12:48:52.055 INFO:root:teuthology version: 1.1.0-471a2f1e 2021-05-20T12:48:52.061 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-05-20T16:47:09.033 INFO:teuthology.results:Tests finished! gathering results... 2021-05-20T16:47:10.674 INFO:scrape:Found 16 jobs 2021-05-20T16:47:10.676 INFO:scrape:Found 1 distinct failure reasons 2021-05-20T16:47:10.676 INFO:scrape:Failure: "2021-05-20 14:48:22.298488 osd.3 (osd.3) 2274 : cluster [ERR] 2.1a deep-scrub : stat mismatch, got 1/1 objects, 0/0 clones, 1/1 dirty, 0/0 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 19/0 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes." in cluster log 2021-05-20T16:47:10.677 INFO:scrape:1 jobs: ['6124168'] 2021-05-20T16:47:10.677 INFO:scrape:suites: ['1-ceph-install/nautilus', '1.1.short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{fsx', '5-finish-upgrade', '7-final-workload/{rbd-python', 'objectstore/default', 'radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw-swift', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'supported-all-distro/ubuntu_latest', 'thrashosds-health}', 'upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack'] 2021-05-20T16:47:10.678 INFO:scrape: 2021-05-20T16:47:10.679 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 15 Fail ================================================================= [6124168] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/ubuntu_latest thrashosds-health} ----------------------------------------------------------------- time: 02:46:48 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124168/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124168/ "2021-05-20 14:48:22.298488 osd.3 (osd.3) 2274 : cluster [ERR] 2.1a deep- scrub : stat mismatch, got 1/1 objects, 0/0 clones, 1/1 dirty, 0/0 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 19/0 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes." in cluster log Pass ================================================================= [6124158] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/centos_latest} time: 01:14:11 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124158/ [6124159] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/centos_latest thrashosds-health} time: 03:40:23 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124159/ [6124160] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/rhel_7 thrashosds-health} time: 02:56:59 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124160/ [6124161] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/ubuntu_16.04 thrashosds-health} time: 02:42:08 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124161/ [6124162] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/rhel_7} time: 01:11:13 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124162/ [6124163] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest thrashosds-health} time: 03:36:37 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124163/ [6124164] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/centos_latest thrashosds-health} time: 03:07:30 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124164/ [6124165] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/rhel_7 thrashosds-health} time: 02:54:17 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124165/ [6124166] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/ubuntu_16.04} time: 01:06:41 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124166/ [6124167] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_16.04 thrashosds-health} time: 03:28:37 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124167/ [6124169] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/centos_latest thrashosds-health} time: 02:52:35 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124169/ [6124170] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/ubuntu_latest} time: 01:05:19 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124170/ [6124171] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/rhel_7 thrashosds-health} time: 03:52:47 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124171/ [6124172] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/ubuntu_16.04 thrashosds-health} time: 02:45:00 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124172/ [6124173] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/ubuntu_latest thrashosds-health} time: 02:43:08 info: http://pulpito.ceph.com/teuthology-2021-05-20_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/6124173/