2022-09-19T19:47:09.488 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18
2022-09-19T19:47:09.490 INFO:teuthology.results:seed: '4273'
2022-09-19T19:47:53.871 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18
2022-09-19T19:47:53.875 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-09-19T20:17:55.690 INFO:teuthology.results:Tests finished! gathering results...
2022-09-19T20:17:55.920 INFO:scrape:Found 6 jobs
2022-09-19T20:17:56.525 INFO:scrape:Found 1 distinct failure reasons
2022-09-19T20:17:56.526 INFO:scrape:Failure: "1663617935.8631663 mds.c (mds.0) 4 : cluster [WRN] Scrub error on dentry [dentry #0x1/subdir [head,head] auth (dversion lock) pv=0 v=1 ino=0x10000000000 state=1073741824 0x5584697c0280] see mds.c log and `damage ls` output for details" in cluster log
2022-09-19T20:17:56.526 INFO:scrape:6 jobs: ['7037798', '7037802', '7037800', '7037799', '7037797', '7037801']
2022-09-19T20:17:56.527 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'osd}', 'overrides/{ignorelist_health', 'tasks/alternate-pool}']
2022-09-19T20:17:56.527 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/alternate-pool}']
2022-09-19T20:17:56.528 INFO:scrape:
2022-09-19T20:17:56.529 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2022-09-19_19:46:59-fs:functional-wip-pdonnell-testing-20220919.174338-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/pdonnell-2022-09-19_19:46:59-fs:functional-wip-pdonnell-testing-20220919.174338-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-09-19_19:46:59-fs:functional-wip-pdonnell-testing-20220919.174338-distro-default-smithi/
failed:  6
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[7037797]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}
-----------------------------------------------------------------
time:   00:19:23
info:   http://pulpito.ceph.com/pdonnell-2022-09-19_19:46:59-fs:functional-wip-pdonnell-testing-20220919.174338-distro-default-smithi/7037797/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-09-19_19:46:59-fs:functional-wip-pdonnell-testing-20220919.174338-distro-default-smithi/7037797/

    "1663618531.669511 mds.b (mds.0) 4 : cluster [WRN] Scrub error on dentry
    [dentry #0x1/subdir [head,head] auth (dversion lock) pv=0 v=1
    ino=0x10000000000 state=1073741824 0x555f36cc2500] see mds.b log and
    `damage ls` output for details" in cluster log

[7037798]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}
-----------------------------------------------------------------
time:   00:13:41
info:   http://pulpito.ceph.com/pdonnell-2022-09-19_19:46:59-fs:functional-wip-pdonnell-testing-20220919.174338-distro-default-smithi/7037798/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-09-19_19:46:59-fs:functional-wip-pdonnell-testing-20220919.174338-distro-default-smithi/7037798/

    "1663617935.8631663 mds.c (mds.0) 4 : cluster [WRN] Scrub error on dentry
    [dentry #0x1/subdir [head,head] auth (dversion lock) pv=0 v=1
    ino=0x10000000000 state=1073741824 0x5584697c0280] see mds.c log and
    `damage ls` output for details" in cluster log

[7037799]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}
-----------------------------------------------------------------
time:   00:08:54
info:   http://pulpito.ceph.com/pdonnell-2022-09-19_19:46:59-fs:functional-wip-pdonnell-testing-20220919.174338-distro-default-smithi/7037799/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-09-19_19:46:59-fs:functional-wip-pdonnell-testing-20220919.174338-distro-default-smithi/7037799/

    "1663617680.354353 mds.a (mds.0) 4 : cluster [WRN] Scrub error on dentry
    [dentry #0x1/subdir [head,head] auth (dversion lock) pv=0 v=1
    ino=0x10000000000 state=1073741824 0x55a71858e500] see mds.a log and
    `damage ls` output for details" in cluster log

[7037800]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}
-----------------------------------------------------------------
time:   00:12:34
info:   http://pulpito.ceph.com/pdonnell-2022-09-19_19:46:59-fs:functional-wip-pdonnell-testing-20220919.174338-distro-default-smithi/7037800/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-09-19_19:46:59-fs:functional-wip-pdonnell-testing-20220919.174338-distro-default-smithi/7037800/

    "1663618100.548443 mds.d (mds.0) 4 : cluster [WRN] Scrub error on dentry
    [dentry #0x1/subdir [head,head] auth (dversion lock) pv=0 v=1
    ino=0x10000000000 state=1073741824 0x55cbd0e4e280] see mds.d log and
    `damage ls` output for details" in cluster log

[7037801]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}
-----------------------------------------------------------------
time:   00:13:21
info:   http://pulpito.ceph.com/pdonnell-2022-09-19_19:46:59-fs:functional-wip-pdonnell-testing-20220919.174338-distro-default-smithi/7037801/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-09-19_19:46:59-fs:functional-wip-pdonnell-testing-20220919.174338-distro-default-smithi/7037801/

    "1663617930.5145905 mds.c (mds.0) 4 : cluster [WRN] Scrub error on dentry
    [dentry #0x1/subdir [head,head] auth (dversion lock) pv=0 v=1
    ino=0x10000000000 state=1073741824 0x55c89ea70500] see mds.c log and
    `damage ls` output for details" in cluster log

[7037802]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}
-----------------------------------------------------------------
time:   00:09:29
info:   http://pulpito.ceph.com/pdonnell-2022-09-19_19:46:59-fs:functional-wip-pdonnell-testing-20220919.174338-distro-default-smithi/7037802/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2022-09-19_19:46:59-fs:functional-wip-pdonnell-testing-20220919.174338-distro-default-smithi/7037802/

    "1663617757.4920805 mds.d (mds.0) 4 : cluster [WRN] Scrub error on dentry
    [dentry #0x1/subdir [head,head] auth (dversion lock) pv=0 v=1
    ino=0x10000000000 state=1073741824 0x557915aae280] see mds.d log and
    `damage ls` output for details" in cluster log