2020-09-21T06:15:00.578 INFO:root:teuthology version: 1.0.0-f390fd3
2020-09-21T06:15:00.579 INFO:teuthology.results:subset: '0/10'
2020-09-21T06:15:00.579 INFO:teuthology.results:seed: '4513'
2020-09-21T13:49:40.560 INFO:root:teuthology version: 1.0.0-f390fd3
2020-09-21T13:49:40.617 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-09-22T01:42:02.391 INFO:teuthology.results:Tests finished! gathering results...
2020-09-22T01:42:29.761 INFO:scrape:Found 259 jobs
2020-09-22T01:42:32.941 INFO:scrape:Found 6 distinct failure reasons
2020-09-22T01:42:32.942 INFO:scrape:Dead: b'2020-09-21T19:39:10.115 DEBUG:teuthology.exit:Finished running handlers'
2020-09-22T01:42:32.942 INFO:scrape:54 jobs
2020-09-22T01:42:32.943 INFO:scrape:suites intersection: ['clusters/{fixed-2', 'openstack}', 'quorum-write/3-size-2-pw-size', 'rados', 'rados:thrash-erasure-code/{ceph', 'thrashosds-health']
2020-09-22T01:42:32.943 INFO:scrape:suites union: ['clusters/{fixed-2', 'fast/fast', 'fast/normal', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'objectstore/filestore-xfs', 'openstack}', 'quorum-write/3-size-2-pw-size', 'rados', 'rados:thrash-erasure-code/{ceph', 'recovery-overrides/{default}', 'recovery-overrides/{more-active-recovery}', 'recovery-overrides/{more-async-partial-recovery}', 'recovery-overrides/{more-async-recovery}', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/careful', 'thrashers/default', 'thrashers/fastread', 'thrashers/minsize_recovery', 'thrashers/morepggrow', 'thrashers/pggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=clay-k=4-m=2}', 'workloads/ec-rados-plugin=jerasure-k=2-m=1}', 'workloads/ec-rados-plugin=jerasure-k=3-m=1}', 'workloads/ec-radosbench}', 'workloads/ec-small-objects-balanced}', 'workloads/ec-small-objects-fast-read}', 'workloads/ec-small-objects-many-deletes}', 'workloads/ec-small-objects}']
2020-09-22T01:42:32.944 INFO:scrape:
2020-09-22T01:42:32.944 INFO:scrape:Failure: Failure object was: {'smithi073.front.sepia.ceph.com': {'msg': "The conditional check 'perform_users_role|bool' failed. The error was: template error while templating string: no filter named 'bool'. String: {% if perform_users_role|bool %} True {% else %} False {% endif %}\n\nThe error appears to be in '/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/users/tasks/create_users.yml': line 10, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Create all admin users with sudo access.\n  ^ here\n", '_ansible_no_log': False}}Traceback (most recent call last):  File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure    log.error(yaml.safe_dump(failure))  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefinedyaml.representer.RepresenterError: ('cannot represent an object', "The conditional check 'perform_users_role|bool' failed. The error was: template error while templating string: no filter named 'bool'. String: {% if perform_users_role|bool %} True {% else %} False {% endif %}\n\nThe error appears to be in '/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/users/tasks/create_users.yml': line 10, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Create all admin users with sudo access.\n  ^ here\n")Failure object was: {'smithi022.front.sepia.ceph.com': {'msg': "The conditional check 'perform_users_role|bool' failed. The error was: template error while templating string: no filter named 'bool'. String: {% if perform_users_role|bool %} True {% else %} False {% endif %}\n\nThe error appears to be in '/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/users/tasks/create_users.yml': line 10, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Create all admin users with sudo access.\n  ^ here\n", '_ansible_no_log': False}}Traceback (most recent call last):  File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure    log.error(yaml.safe_dump(failure))  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefinedyaml.representer.RepresenterError: ('cannot represent an object', "The conditional check 'perform_users_role|bool' failed. The error was: template error while templating string: no filter named 'bool'. String: {% if perform_users_role|bool %} True {% else %} False {% endif %}\n\nThe error appears to be in '/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/users/tasks/create_users.yml': line 10, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Create all admin users with sudo access.\n  ^ here\n")
2020-09-22T01:42:32.944 INFO:scrape:1 jobs: ['5455714']
2020-09-22T01:42:32.945 INFO:scrape:suites: ['clusters/{fixed-2', 'fast/fast', 'msgr-failures/few', 'objectstore/bluestore-comp-lz4', 'openstack}', 'quorum-write/3-size-2-pw-size', 'rados', 'rados:thrash-erasure-code/{ceph', 'recovery-overrides/{more-active-recovery}', 'supported-random-distro$/{rhel_8}', 'thrashers/fastread', 'thrashosds-health', 'workloads/ec-rados-plugin=clay-k=4-m=2}']
2020-09-22T01:42:32.945 INFO:scrape:
2020-09-22T01:42:32.946 INFO:scrape:Failure: reached maximum tries (800) after waiting for 4800 seconds
2020-09-22T01:42:32.946 INFO:scrape:25 jobs: ['5455541', '5455613', '5455757', '5455645', '5455717', '5455693', '5455621', '5455725', '5455589', '5455733', '5455653', '5455701', '5455605', '5455629', '5455581', '5455677', '5455669', '5455549', '5455741', '5455565', '5455525', '5455597', '5455637', '5455749', '5455517']
2020-09-22T01:42:32.946 INFO:scrape:suites intersection: ['clusters/{fixed-2', 'fast/normal', 'openstack}', 'quorum-write/3-size-2-pw-size', 'rados', 'rados:thrash-erasure-code/{ceph', 'thrashosds-health', 'workloads/ec-radosbench}']
2020-09-22T01:42:32.947 INFO:scrape:suites union: ['clusters/{fixed-2', 'fast/normal', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'objectstore/filestore-xfs', 'openstack}', 'quorum-write/3-size-2-pw-size', 'rados', 'rados:thrash-erasure-code/{ceph', 'recovery-overrides/{default}', 'recovery-overrides/{more-active-recovery}', 'recovery-overrides/{more-async-partial-recovery}', 'recovery-overrides/{more-async-recovery}', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/careful', 'thrashers/default', 'thrashers/fastread', 'thrashers/morepggrow', 'thrashers/pggrow', 'thrashosds-health', 'workloads/ec-radosbench}']
2020-09-22T01:42:32.947 INFO:scrape:
2020-09-22T01:42:32.947 INFO:scrape:Failure: SSH connection to smithi101 was lost: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 20 --max-in-flight 8 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 0 --op write 0 --op delete 20 --op append 5 --op write_excl 0 --op append_excl 5 --pool unique_pool_0'
2020-09-22T01:42:32.947 INFO:scrape:1 jobs: ['5455760']
2020-09-22T01:42:32.948 INFO:scrape:suites: ['clusters/{fixed-2', 'fast/fast', 'msgr-failures/osd-delay', 'objectstore/bluestore-comp-snappy', 'openstack}', 'quorum-write/3-size-2-pw-size', 'rados', 'rados:thrash-erasure-code/{ceph', 'recovery-overrides/{default}', 'supported-random-distro$/{centos_8}', 'thrashers/pggrow', 'thrashosds-health', 'workloads/ec-small-objects-many-deletes}']
2020-09-22T01:42:32.948 INFO:scrape:
2020-09-22T01:42:32.948 INFO:scrape:Failure: psutil.NoSuchProcess process no longer exists (pid=18722)
2020-09-22T01:42:32.949 INFO:scrape:4 jobs: ['5455610', '5455559', '5455732', '5455731']
2020-09-22T01:42:32.949 INFO:scrape:suites intersection: ['clusters/{fixed-2', 'openstack}', 'quorum-write/3-size-2-pw-size', 'rados', 'rados:thrash-erasure-code/{ceph', 'thrashosds-health']
2020-09-22T01:42:32.949 INFO:scrape:suites union: ['clusters/{fixed-2', 'fast/fast', 'fast/normal', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-hybrid', 'objectstore/filestore-xfs', 'openstack}', 'quorum-write/3-size-2-pw-size', 'rados', 'rados:thrash-erasure-code/{ceph', 'recovery-overrides/{default}', 'recovery-overrides/{more-async-recovery}', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/careful', 'thrashers/default', 'thrashers/pggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=clay-k=4-m=2}', 'workloads/ec-rados-plugin=jerasure-k=2-m=1}', 'workloads/ec-rados-plugin=jerasure-k=3-m=1}', 'workloads/ec-small-objects-fast-read}']
2020-09-22T01:42:32.950 INFO:scrape:
2020-09-22T01:42:32.950 INFO:scrape:Dead: b'2020-09-22T01:15:43.654 INFO:tasks.ceph.osd.0.smithi164.stderr:2020-09-22T01:15:43.652+0000 7fab8ccca700 -1 received  signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0  (PID: 32658) UID: 0'
2020-09-22T01:42:32.951 INFO:scrape:1 jobs: ['5455754']
2020-09-22T01:42:32.951 INFO:scrape:suites: ['clusters/{fixed-2', 'fast/fast', 'msgr-failures/osd-delay', 'objectstore/bluestore-hybrid', 'openstack}', 'quorum-write/3-size-2-pw-size', 'rados', 'rados:thrash-erasure-code/{ceph', 'recovery-overrides/{more-active-recovery}', 'supported-random-distro$/{rhel_8}', 'thrashers/pggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=clay-k=4-m=2}']
2020-09-22T01:42:32.951 INFO:scrape: