2021-03-31T01:31:54.263 INFO:root:teuthology version: 1.1.0-9870012c 2021-03-31T01:31:54.265 INFO:teuthology.results:seed: '8070' 2021-03-31T01:45:32.851 INFO:root:teuthology version: 1.1.0-9870012c 2021-03-31T01:45:32.883 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-03-31T05:43:33.901 INFO:teuthology.results:Tests finished! gathering results... 2021-03-31T05:43:37.438 INFO:scrape:Found 16 jobs 2021-03-31T05:43:38.046 INFO:scrape:Found 3 distinct failure reasons 2021-03-31T05:43:38.047 INFO:scrape:Failure: {'smithi081.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi200.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi008.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} 2021-03-31T05:43:38.048 INFO:scrape:3 jobs: ['5996160', '5996158', '5996169'] 2021-03-31T05:43:38.048 INFO:scrape:suites intersection: [] 2021-03-31T05:43:38.049 INFO:scrape:suites union: ['1-ceph-install/nautilus', '1.1.short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{fsx', '5-finish-upgrade', '7-final-workload/{rbd-python', 'objectstore/bluestore-bitmap', 'objectstore/default', 'radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw-swift', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'supported-all-distro/rhel_7', 'supported-all-distro/rhel_7}', 'thrashosds-health}', 'upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade', 'upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack'] 2021-03-31T05:43:38.049 INFO:scrape: 2021-03-31T05:43:38.050 INFO:scrape:Failure: timed out waiting for admin_socket to appear after osd.4 restart 2021-03-31T05:43:38.050 INFO:scrape:3 jobs: ['5996162', '5996166', '5996170'] 2021-03-31T05:43:38.051 INFO:scrape:suites intersection: ['1-ceph-install/nautilus', '1.1.short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{fsx', '5-finish-upgrade', '7-final-workload/{rbd-python', 'objectstore/default', 'radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw-swift', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health}', 'upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack'] 2021-03-31T05:43:38.051 INFO:scrape:suites union: ['1-ceph-install/nautilus', '1.1.short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{fsx', '5-finish-upgrade', '7-final-workload/{rbd-python', 'objectstore/default', 'radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw-swift', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'supported-all-distro/centos_latest', 'supported-all-distro/ubuntu_16.04', 'supported-all-distro/ubuntu_latest', 'thrashosds-health}', 'upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack'] 2021-03-31T05:43:38.052 INFO:scrape: 2021-03-31T05:43:38.052 INFO:scrape:Failure: {'smithi166.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi041.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi080.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} 2021-03-31T05:43:38.053 INFO:scrape:1 jobs: ['5996163'] 2021-03-31T05:43:38.053 INFO:scrape:suites: ['1-ceph-install/nautilus', '1.1.short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{fsx', '5-finish-upgrade', '7-final-workload/{rbd-python', 'objectstore/filestore-xfs', 'radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw-swift', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'supported-all-distro/rhel_7', 'thrashosds-health}', 'upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack'] 2021-03-31T05:43:38.054 INFO:scrape: 2021-03-31T05:43:38.056 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/ failed: 3 dead: 4 running: 0 waiting: 0 queued: 0 passed: 9 Fail ================================================================= [5996162] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/centos_latest thrashosds-health} ----------------------------------------------------------------- time: 02:07:48 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996162/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996162/ sentry: https://sentry.ceph.com/organizations/ceph/?query=15cb2eb5da2b458d89c9c80a013bd1a2 timed out waiting for admin_socket to appear after osd.4 restart [5996166] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/ubuntu_latest thrashosds-health} ----------------------------------------------------------------- time: 01:43:13 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996166/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996166/ sentry: https://sentry.ceph.com/organizations/ceph/?query=28f8717175ec4eb396c9d0e704fd64a4 timed out waiting for admin_socket to appear after osd.7 restart [5996170] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/ubuntu_16.04 thrashosds-health} ----------------------------------------------------------------- time: 02:20:12 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996170/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996170/ sentry: https://sentry.ceph.com/organizations/ceph/?query=44aa1d15602744bb8d65eff7508080b2 timed out waiting for admin_socket to appear after osd.1 restart Dead ================================================================= [5996158] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/rhel_7 thrashosds-health} ----------------------------------------------------------------- time: 00:03:44 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996158/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996158/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fd18143b69c04048aa71dbd4f8a27402 {'smithi110.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi089.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi075.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [5996160] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/rhel_7} ----------------------------------------------------------------- time: 00:03:46 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996160/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996160/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5a2cd8893f204338b0c9bac344b74b0b {'smithi081.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi200.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi008.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [5996163] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/rhel_7 thrashosds-health} ----------------------------------------------------------------- time: 00:03:59 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996163/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996163/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0689e13a5a204af78dc93b263c767479 {'smithi166.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi041.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi080.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} [5996169] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/rhel_7 thrashosds-health} ----------------------------------------------------------------- time: 00:03:40 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996169/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996169/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9b627904b6c045f6aefd11146f735044 {'smithi102.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi096.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi141.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} Pass ================================================================= [5996156] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/centos_latest} time: 01:10:25 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996156/ [5996157] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/centos_latest thrashosds-health} time: 03:50:08 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996157/ [5996159] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/ubuntu_16.04 thrashosds-health} time: 02:57:12 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996159/ [5996161] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest thrashosds-health} time: 03:51:53 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996161/ [5996164] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/ubuntu_16.04} time: 01:06:36 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996164/ [5996165] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_16.04 thrashosds-health} time: 03:40:16 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996165/ [5996167] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/centos_latest thrashosds-health} time: 02:39:14 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996167/ [5996168] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/ubuntu_latest} time: 01:06:11 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996168/ [5996171] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/ubuntu_latest thrashosds-health} time: 02:48:13 info: http://pulpito.ceph.com/teuthology-2021-03-25_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5996171/