2021-03-08T01:55:07.018 INFO:root:teuthology version: 1.1.0-3082387b
2021-03-08T01:55:07.020 INFO:teuthology.results:seed: '1224'
2021-03-08T10:18:15.284 INFO:root:teuthology version: 1.1.0-3082387b
2021-03-08T10:18:15.330 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-03-08T14:18:33.170 INFO:teuthology.results:Tests finished! gathering results...
2021-03-08T14:18:35.894 INFO:scrape:Found 16 jobs
2021-03-08T14:18:35.900 INFO:scrape:Found 2 distinct failure reasons
2021-03-08T14:18:35.900 INFO:scrape:Failure: failed to recover before timeout expired
2021-03-08T14:18:35.901 INFO:scrape:3 jobs: ['5931747', '5931735', '5931739']
2021-03-08T14:18:35.901 INFO:scrape:suites intersection: ['1-ceph-install/nautilus', '1.1.short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{fsx', '5-finish-upgrade', '7-final-workload/{rbd-python', 'objectstore/default', 'radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw-swift', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health}', 'upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack']
2021-03-08T14:18:35.902 INFO:scrape:suites union: ['1-ceph-install/nautilus', '1.1.short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{fsx', '5-finish-upgrade', '7-final-workload/{rbd-python', 'objectstore/default', 'radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw-swift', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'supported-all-distro/centos_latest', 'supported-all-distro/rhel_7', 'supported-all-distro/ubuntu_16.04', 'thrashosds-health}', 'upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack']
2021-03-08T14:18:35.902 INFO:scrape:
2021-03-08T14:18:35.902 INFO:scrape:Failure: timed out waiting for admin_socket to appear after osd.5 restart
2021-03-08T14:18:35.903 INFO:scrape:1 jobs: ['5931743']
2021-03-08T14:18:35.903 INFO:scrape:suites: ['1-ceph-install/nautilus', '1.1.short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{fsx', '5-finish-upgrade', '7-final-workload/{rbd-python', 'objectstore/default', 'radosbench', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw-swift', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'supported-all-distro/ubuntu_latest', 'thrashosds-health}', 'upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack']
2021-03-08T14:18:35.904 INFO:scrape:
2021-03-08T14:18:35.905 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/
failed:  4
dead:    0
running: 0
waiting: 0
queued:  0
passed:  12


Fail
=================================================================
[5931735]  upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/rhel_7 thrashosds-health}
-----------------------------------------------------------------
time:   02:13:21
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931735/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931735/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e55890ec2c6f4fc28abfc01cb4be2fce

    failed to recover before timeout expired

[5931739]  upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/centos_latest thrashosds-health}
-----------------------------------------------------------------
time:   02:09:09
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931739/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931739/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1f231ed0d0d94c9aadf545cbf7bbd738

    failed to recover before timeout expired

[5931743]  upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/ubuntu_latest thrashosds-health}
-----------------------------------------------------------------
time:   01:47:44
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931743/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931743/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fa379954addb47c3b49cbf8e4b86f067

    timed out waiting for admin_socket to appear after osd.5 restart

[5931747]  upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/default supported-all-distro/ubuntu_16.04 thrashosds-health}
-----------------------------------------------------------------
time:   02:33:04
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931747/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931747/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e81df5925ec643669ce00b9493af732a

    failed to recover before timeout expired



Pass
=================================================================
[5931733] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/centos_latest}
time:   01:26:58
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931733/

[5931734] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/centos_latest thrashosds-health}
time:   04:23:59
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931734/

[5931736] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/ubuntu_16.04 thrashosds-health}
time:   03:32:32
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931736/

[5931737] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/rhel_7}
time:   01:40:37
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931737/

[5931738] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_latest thrashosds-health}
time:   04:11:25
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931738/

[5931740] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/rhel_7 thrashosds-health}
time:   03:22:14
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931740/

[5931741] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/ubuntu_16.04}
time:   01:37:04
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931741/

[5931742] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/ubuntu_16.04 thrashosds-health}
time:   03:43:06
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931742/

[5931744] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/centos_latest thrashosds-health}
time:   03:21:39
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931744/

[5931745] upgrade:nautilus-p2p/nautilus-p2p-parallel/{point-to-point-upgrade supported-all-distro/ubuntu_latest}
time:   01:22:39
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931745/

[5931746] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/rhel_7 thrashosds-health}
time:   03:56:17
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931746/

[5931748] upgrade:nautilus-p2p/nautilus-p2p-stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1.short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{fsx radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 7-final-workload/{rbd-python rgw-swift snaps-many-objects} objectstore/filestore-xfs supported-all-distro/ubuntu_latest thrashosds-health}
time:   03:25:27
info:   http://pulpito.ceph.com/teuthology-2021-03-04_01:25:03-upgrade:nautilus-p2p-nautilus-distro-basic-smithi/5931748/