2021-03-23T21:13:28.027 INFO:root:teuthology version: 1.1.0-6b3150e9 2021-03-23T21:13:28.028 INFO:teuthology.results:seed: '6332' 2021-03-24T02:58:20.847 INFO:root:teuthology version: 1.1.0-6b3150e9 2021-03-24T02:58:20.874 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-03-24T05:43:33.379 INFO:teuthology.results:Tests finished! gathering results... 2021-03-24T05:43:37.859 INFO:scrape:Found 32 jobs 2021-03-24T05:43:37.987 INFO:scrape:Found 2 distinct failure reasons 2021-03-24T05:43:37.988 INFO:scrape:Failure: Command failed (workunit test rbd/test_librbd_python.sh) on smithi174 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=08ab4ae8836f18f5631f6d91fdac3d264a7027ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd_python.sh' 2021-03-24T05:43:37.988 INFO:scrape:8 jobs: ['5973981', '5973985', '5973977', '5973989', '5973969', '5973973', '5973993', '5973965'] 2021-03-24T05:43:37.989 INFO:scrape:suites intersection: ['1-ceph-install/nautilus', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{radosbench', '5-finish-upgrade', '6-octopus', '7-msgr2', '8-final-workload/{rbd-python', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_18.04}', 'upgrade:nautilus-x/stress-split/{0-cluster/{openstack'] 2021-03-24T05:43:37.990 INFO:scrape:suites union: ['1-ceph-install/nautilus', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{radosbench', '5-finish-upgrade', '6-octopus', '7-msgr2', '8-final-workload/{rbd-python', 'mon_election/classic', 'mon_election/connectivity', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_18.04}', 'upgrade:nautilus-x/stress-split/{0-cluster/{openstack'] 2021-03-24T05:43:37.990 INFO:scrape: 2021-03-24T05:43:37.991 INFO:scrape:Failure: {'Failure object was': {'smithi059.front.sepia.ceph.com': {'msg': 'Failed to checkout branch master', 'stdout': '', 'stderr': "fatal: Unable to create '/home/teuthworker/.cache/src/keys/.git/index.lock': File exists.\n\nAnother git process seems to be running in this repository, e.g.\nan editor opened by 'git commit'. Please make sure all processes\nare terminated then try again. If it still fails, a git process\nmay have crashed in this repository earlier:\nremove the file manually to continue.\n", 'rc': 128, 'invocation': {'module_args': {'repo': 'https://github.com/ceph/keys', 'version': 'master', 'force': True, 'dest': '/home/teuthworker/.cache/src/keys', 'remote': 'origin', 'clone': True, 'update': True, 'verify_commit': False, 'gpg_whitelist': [], 'accept_hostkey': False, 'bare': False, 'recursive': True, 'track_submodules': False, 'refspec': 'None', 'reference': 'None', 'depth': 'None', 'key_file': 'None', 'ssh_opts': 'None', 'executable': 'None', 'umask': 'None', 'archive': 'None', 'separate_git_dir': 'None'}}, 'stdout_lines': [], 'stderr_lines': ["fatal: Unable to create '/home/teuthworker/.cache/src/keys/.git/index.lock': File exists.", '', 'Another git process seems to be running in this repository, e.g.', "an editor opened by 'git commit'. Please make sure all processes", 'are terminated then try again. If it still fails, a git process', 'may have crashed in this repository earlier:', 'remove the file manually to continue.'], '_ansible_no_log': False, 'attempts': 5, 'changed': False, '_ansible_delegated_vars': {}}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', '_ansible_no_log')"} 2021-03-24T05:43:37.991 INFO:scrape:1 jobs: ['5973995'] 2021-03-24T05:43:37.992 INFO:scrape:suites: ['1-ceph-install/nautilus', '1.1-pg-log-overrides/normal_pg_log', '2-workload/{blogbench', '3-upgrade-sequence/upgrade-all', '4-octopus', '5-final-workload/{blogbench', 'ec-rados-default', 'mon_election/classic', 'objectstore/filestore-xfs', 'rados-snaps-few-objects', 'rados_api', 'rados_loadgenbig', 'rados_loadgenmix', 'rados_mon_thrash', 'rbd_cls', 'rbd_import_export', 'rgw', 'rgw_ragweed_check}', 'rgw_ragweed_prepare', 'start}', 'test_rbd_api', 'test_rbd_python}', 'ubuntu_18.04}', 'upgrade:nautilus-x/parallel/{0-cluster/{openstack'] 2021-03-24T05:43:37.992 INFO:scrape: 2021-03-24T05:43:37.996 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/ failed: 8 dead: 1 running: 0 waiting: 0 queued: 0 passed: 23 Fail ================================================================= [5973965] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/bluestore-bitmap thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 02:29:47 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973965/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973965/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d3067e6d37bd4508a3814353595080bd Command failed (workunit test rbd/test_librbd_python.sh) on smithi197 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=08ab4ae8836f18f5631f6d91fdac3d264a7027ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rbd/test_librbd_python.sh' [5973969] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/filestore-xfs thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 02:22:28 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973969/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973969/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d0b0c096acc5443cad9d26e48b02f327 Command failed (workunit test rbd/test_librbd_python.sh) on smithi201 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=08ab4ae8836f18f5631f6d91fdac3d264a7027ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rbd/test_librbd_python.sh' [5973973] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/bluestore-bitmap thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 02:37:55 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973973/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973973/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0e685b96fef34b6fba3778e1f18529d1 Command failed (workunit test rbd/test_librbd_python.sh) on smithi142 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=08ab4ae8836f18f5631f6d91fdac3d264a7027ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rbd/test_librbd_python.sh' [5973977] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/filestore-xfs thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 02:29:54 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973977/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973977/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2bba869b76ac4527821935aad15ea42c Command failed (workunit test rbd/test_librbd_python.sh) on smithi175 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=08ab4ae8836f18f5631f6d91fdac3d264a7027ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rbd/test_librbd_python.sh' [5973981] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/bluestore-bitmap thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 02:48:51 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973981/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973981/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a54cf913c8234292ab5c9d43cd6b2e4f Command failed (workunit test rbd/test_librbd_python.sh) on smithi174 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=08ab4ae8836f18f5631f6d91fdac3d264a7027ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rbd/test_librbd_python.sh' [5973985] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/filestore-xfs thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 02:51:43 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973985/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973985/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fae5a2ff42cb4e458f06e5d7cfd44100 Command failed (workunit test rbd/test_librbd_python.sh) on smithi184 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=08ab4ae8836f18f5631f6d91fdac3d264a7027ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rbd/test_librbd_python.sh' [5973989] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/bluestore-bitmap thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 02:36:18 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973989/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973989/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3220c1ab00434b6fab23456966ce90f9 Command failed (workunit test rbd/test_librbd_python.sh) on smithi159 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=08ab4ae8836f18f5631f6d91fdac3d264a7027ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rbd/test_librbd_python.sh' [5973993] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/filestore-xfs thrashosds-health ubuntu_18.04} ----------------------------------------------------------------- time: 02:34:03 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973993/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973993/ sentry: https://sentry.ceph.com/organizations/ceph/?query=12593f4fcf7544f79af8146158ffd992 Command failed (workunit test rbd/test_librbd_python.sh) on smithi175 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=08ab4ae8836f18f5631f6d91fdac3d264a7027ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rbd/test_librbd_python.sh' Dead ================================================================= [5973995] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/classic objectstore/filestore-xfs ubuntu_18.04} ----------------------------------------------------------------- time: 00:02:14 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973995/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973995/ sentry: https://sentry.ceph.com/organizations/ceph/?query=74254e6cdb2d4fd88b7fd347a66f6b58 {'Failure object was': {'smithi059.front.sepia.ceph.com': {'msg': 'Failed to checkout branch master', 'stdout': '', 'stderr': "fatal: Unable to create '/home/teuthworker/.cache/src/keys/.git/index.lock': File exists.\n\nAnother git process seems to be running in this repository, e.g.\nan editor opened by 'git commit'. Please make sure all processes\nare terminated then try again. If it still fails, a git process\nmay have crashed in this repository earlier:\nremove the file manually to continue.\n", 'rc': 128, 'invocation': {'module_args': {'repo': 'https://github.com/ceph/keys', 'version': 'master', 'force': True, 'dest': '/home/teuthworker/.cache/src/keys', 'remote': 'origin', 'clone': True, 'update': True, 'verify_commit': False, 'gpg_whitelist': [], 'accept_hostkey': False, 'bare': False, 'recursive': True, 'track_submodules': False, 'refspec': 'None', 'reference': 'None', 'depth': 'None', 'key_file': 'None', 'ssh_opts': 'None', 'executable': 'None', 'umask': 'None', 'archive': 'None', 'separate_git_dir': 'None'}}, 'stdout_lines': [], 'stderr_lines': ["fatal: Unable to create '/home/teuthworker/.cache/src/keys/.git/index.lock': File exists.", '', 'Another git process seems to be running in this repository, e.g.', "an editor opened by 'git commit'. Please make sure all processes", 'are terminated then try again. If it still fails, a git process', 'may have crashed in this repository earlier:', 'remove the file manually to continue.'], '_ansible_no_log': False, 'attempts': 5, 'changed': False, '_ansible_delegated_vars': {}}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15e ae4604f361990ea65737b28dff1/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15e ae4604f361990ea65737b28dff1/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', '_ansible_no_log')"} Pass ================================================================= [5973964] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/classic objectstore/bluestore-bitmap ubuntu_18.04} time: 02:15:18 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973964/ [5973966] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04} time: 01:58:45 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973966/ [5973967] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/connectivity objectstore/filestore-xfs ubuntu_18.04} time: 02:06:52 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973967/ [5973968] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/connectivity objectstore/bluestore-bitmap ubuntu_18.04} time: 02:12:38 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973968/ [5973970] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04} time: 01:23:38 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973970/ [5973971] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/classic objectstore/filestore-xfs ubuntu_18.04} time: 02:08:30 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973971/ [5973972] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/classic objectstore/bluestore-bitmap ubuntu_18.04} time: 02:14:05 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973972/ [5973974] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04} time: 01:39:00 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973974/ [5973975] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/connectivity objectstore/filestore-xfs ubuntu_18.04} time: 02:08:39 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973975/ [5973976] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/connectivity objectstore/bluestore-bitmap ubuntu_18.04} time: 02:13:51 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973976/ [5973978] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04} time: 01:46:59 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973978/ [5973979] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/classic objectstore/filestore-xfs ubuntu_18.04} time: 02:55:39 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973979/ [5973980] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/classic objectstore/bluestore-bitmap ubuntu_18.04} time: 02:12:27 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973980/ [5973982] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04} time: 01:43:14 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973982/ [5973983] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/connectivity objectstore/filestore-xfs ubuntu_18.04} time: 02:17:41 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973983/ [5973984] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/connectivity objectstore/bluestore-bitmap ubuntu_18.04} time: 02:14:42 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973984/ [5973986] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04} time: 01:45:58 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973986/ [5973987] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/classic objectstore/filestore-xfs ubuntu_18.04} time: 02:06:13 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973987/ [5973988] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/classic objectstore/bluestore-bitmap ubuntu_18.04} time: 02:14:42 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973988/ [5973990] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload mon_election/classic thrashosds-health ubuntu_18.04} time: 01:24:36 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973990/ [5973991] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/connectivity objectstore/filestore-xfs ubuntu_18.04} time: 02:07:02 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973991/ [5973992] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} mon_election/connectivity objectstore/bluestore-bitmap ubuntu_18.04} time: 02:13:06 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973992/ [5973994] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload mon_election/connectivity thrashosds-health ubuntu_18.04} time: 01:57:05 info: http://pulpito.ceph.com/teuthology-2021-03-17_14:23:03-upgrade:nautilus-x-pacific-distro-basic-smithi/5973994/