2020-05-25T14:24:19.442 INFO:root:teuthology version: 1.0.0-4413d31 2020-05-25T14:24:19.445 INFO:teuthology.results:seed: '3101' 2020-05-25T14:32:45.759 INFO:root:teuthology version: 1.0.0-4413d31 2020-05-25T14:32:45.770 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-05-26T00:28:57.779 INFO:teuthology.results:Tests finished! gathering results... 2020-05-26T00:28:57.841 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/ failed: 4 dead: 0 running: 0 waiting: 0 queued: 0 passed: 12 Fail ================================================================= [5091688] upgrade:nautilus-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-msgr2.yaml 8-final-workload/{rbd-python.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml thrashosds-health.yaml ubuntu_latest.yaml} ----------------------------------------------------------------- time: 02:33:58 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091688/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091688/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=c11dbbd5f17742748196abf45d730475 Command failed on smithi018 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5091692] upgrade:nautilus-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-msgr2.yaml 8-final-workload/{rbd-python.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml thrashosds-health.yaml ubuntu_latest.yaml} ----------------------------------------------------------------- time: 02:06:55 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091692/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091692/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=85fef2f3421b46d4a1a0fbc55e020244 Command failed on smithi043 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5091696] upgrade:nautilus-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-msgr2.yaml 8-final-workload/{rbd-python.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml thrashosds-health.yaml ubuntu_latest.yaml} ----------------------------------------------------------------- time: 02:40:17 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091696/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091696/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=4236fa9563404fa493626bdf19ee22fc Command failed on smithi133 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5091700] upgrade:nautilus-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-msgr2.yaml 8-final-workload/{rbd-python.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml thrashosds-health.yaml ubuntu_latest.yaml} ----------------------------------------------------------------- time: 02:29:45 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091700/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091700/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=2d8fca5b92554ca18ee8d41dd82cda07 Command failed on smithi160 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' Pass ================================================================= [5091687] upgrade:nautilus-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-octopus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml} objectstore/bluestore-bitmap.yaml ubuntu_latest.yaml} time: 02:26:44 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091687/ [5091689] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-nautilus-install/nautilus.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 3.1-objectstore/bluestore-bitmap.yaml 4-ec-workload/{rados-ec-workload.yaml rbd-ec-workload.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-final-workload.yaml thrashosds-health.yaml ubuntu_latest.yaml} time: 02:15:51 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091689/ [5091690] upgrade:nautilus-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-octopus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml} objectstore/filestore-xfs.yaml ubuntu_latest.yaml} time: 02:21:52 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091690/ [5091691] upgrade:nautilus-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-octopus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml} objectstore/bluestore-bitmap.yaml ubuntu_latest.yaml} time: 02:26:40 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091691/ [5091693] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-nautilus-install/nautilus.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 3.1-objectstore/filestore-xfs.yaml 4-ec-workload/{rados-ec-workload.yaml rbd-ec-workload.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-final-workload.yaml thrashosds-health.yaml ubuntu_latest.yaml} time: 01:45:27 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091693/ [5091694] upgrade:nautilus-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-octopus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml} objectstore/filestore-xfs.yaml ubuntu_latest.yaml} time: 02:21:02 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091694/ [5091695] upgrade:nautilus-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-octopus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml} objectstore/bluestore-bitmap.yaml ubuntu_latest.yaml} time: 02:24:41 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091695/ [5091697] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-nautilus-install/nautilus.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 3.1-objectstore/bluestore-bitmap.yaml 4-ec-workload/{rados-ec-workload.yaml rbd-ec-workload.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-final-workload.yaml thrashosds-health.yaml ubuntu_latest.yaml} time: 02:01:06 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091697/ [5091698] upgrade:nautilus-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-octopus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml} objectstore/filestore-xfs.yaml ubuntu_latest.yaml} time: 02:24:54 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091698/ [5091699] upgrade:nautilus-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-octopus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml} objectstore/bluestore-bitmap.yaml ubuntu_latest.yaml} time: 02:27:37 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091699/ [5091701] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-nautilus-install/nautilus.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 3.1-objectstore/filestore-xfs.yaml 4-ec-workload/{rados-ec-workload.yaml rbd-ec-workload.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-final-workload.yaml thrashosds-health.yaml ubuntu_latest.yaml} time: 02:02:08 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091701/ [5091702] upgrade:nautilus-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-octopus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml} objectstore/filestore-xfs.yaml ubuntu_latest.yaml} time: 02:21:16 info: http://pulpito.ceph.com/teuthology-2020-05-25_14:23:03-upgrade:nautilus-x-octopus-distro-basic-smithi/5091702/ 2020-05-26T00:28:58.026 INFO:teuthology.results:starting coverage generation 2020-05-26T00:28:58.032 ERROR:teuthology.results:error generating memo/results Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 35, in main int(args['--timeout']), args['--dry-run']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 84, in results generate_coverage(archive_dir, name) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 107, in generate_coverage archive_dir, File "/usr/lib/python3.6/subprocess.py", line 729, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage': '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage'