2020-09-16T05:41:45.564 INFO:root:teuthology version: 1.0.0-55bd2dc 2020-09-16T05:41:45.565 INFO:teuthology.results:seed: '5169' 2020-09-16T05:47:48.642 INFO:root:teuthology version: 1.0.0-55bd2dc 2020-09-16T05:47:48.670 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-09-16T10:15:54.586 INFO:teuthology.results:Tests finished! gathering results... 2020-09-16T10:15:58.966 INFO:scrape:Found 16 jobs 2020-09-16T10:16:00.153 INFO:scrape:Found 2 distinct failure reasons 2020-09-16T10:16:00.155 INFO:scrape:Failure: timed out waiting for admin_socket to appear after osd.0 restart 2020-09-16T10:16:00.156 INFO:scrape:2 jobs: ['5419627', '5419622'] 2020-09-16T10:16:00.156 INFO:scrape:suites intersection: ['1-ceph-install/nautilus', '1.1.short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{fsx', '5-finish-upgrade', '7-final-workload/{rbd-python', 'radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw-swift', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'supported-all-distro/rhel_7', 'thrashosds-health}', 'upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack'] 2020-09-16T10:16:00.156 INFO:scrape:suites union: ['1-ceph-install/nautilus', '1.1.short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{fsx', '5-finish-upgrade', '7-final-workload/{rbd-python', 'objectstore/default', 'objectstore/filestore-xfs', 'radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw-swift', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'supported-all-distro/rhel_7', 'thrashosds-health}', 'upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack'] 2020-09-16T10:16:00.157 INFO:scrape: 2020-09-16T10:16:00.157 INFO:scrape:Failure: "2020-09-16 06:21:31.273997 mds.a (mds.0) 1 : cluster [WRN] evicting unresponsive client smithi062:x (7677), after 301.04 seconds" in cluster log 2020-09-16T10:16:00.158 INFO:scrape:1 jobs: ['5419632'] 2020-09-16T10:16:00.158 INFO:scrape:suites: ['supported-all-distro/ubuntu_latest}', 'upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade'] 2020-09-16T10:16:00.159 INFO:scrape: 2020-09-16T10:16:00.161 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/ failed: 3 dead: 0 running: 0 waiting: 0 queued: 0 passed: 13 Fail ================================================================= [5419622] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/rhel_7 thrashosds-health} ----------------------------------------------------------------- time: 03:04:41 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419622/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419622/ sentry: https://sentry.ceph.com/sepia/teuthology/?query=d22c80d0d5fd4a4ca6143fff1f709fe8 timed out waiting for admin_socket to appear after osd.3 restart [5419627] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/rhel_7 thrashosds-health} ----------------------------------------------------------------- time: 02:50:21 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419627/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419627/ sentry: https://sentry.ceph.com/sentry/sepia/?query=d5aa8e554a6543baa15d4090add3a66f timed out waiting for admin_socket to appear after osd.0 restart [5419632] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/ubuntu_latest} ----------------------------------------------------------------- time: 01:06:42 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419632/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419632/ "2020-09-16 06:21:31.273997 mds.a (mds.0) 1 : cluster [WRN] evicting unresponsive client smithi062:x (7677), after 301.04 seconds" in cluster log Pass ================================================================= [5419619] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/centos_latest} time: 01:15:15 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419619/ [5419621] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/centos_latest thrashosds-health} time: 03:49:55 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419621/ [5419623] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/ubuntu_16.04 thrashosds-health} time: 03:21:40 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419623/ [5419624] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/rhel_7} time: 01:12:28 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419624/ [5419625] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest thrashosds-health} time: 03:57:22 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419625/ [5419626] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/centos_latest thrashosds-health} time: 03:16:30 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419626/ [5419628] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/ubuntu_16.04} time: 01:06:42 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419628/ [5419629] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_16.04 thrashosds-health} time: 03:57:24 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419629/ [5419630] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/ubuntu_latest thrashosds-health} time: 03:19:31 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419630/ [5419631] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/centos_latest thrashosds-health} time: 03:28:08 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419631/ [5419633] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/rhel_7 thrashosds-health} time: 03:44:52 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419633/ [5419634] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/ubuntu_16.04 thrashosds-health} time: 03:21:42 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419634/ [5419635] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/ubuntu_latest thrashosds-health} time: 03:31:05 info: http://pulpito.ceph.com/teuthology-2020-09-10_01:25:02-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5419635/