2022-06-22T12:04:32.854 INFO:root:teuthology version: 1.1.1.dev374+g31a356e0 2022-06-22T12:04:32.855 INFO:teuthology.results:seed: '5474' 2022-06-22T12:06:42.719 INFO:root:teuthology version: 1.1.1.dev374+g31a356e0 2022-06-22T12:06:42.725 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-06-22T13:03:47.891 INFO:teuthology.results:Tests finished! gathering results... 2022-06-22T13:03:48.337 INFO:scrape:Found 5 jobs 2022-06-22T13:03:53.453 INFO:scrape:Found 3 distinct failure reasons 2022-06-22T13:03:53.454 INFO:scrape:Failure: Test failure: test_dump_inmemory_log_on_missed_internal_heartbeats (tasks.cephfs.test_misc.TestMisc) 2022-06-22T13:03:53.454 INFO:scrape:2 jobs: ['6892119', '6892121'] 2022-06-22T13:03:53.455 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1-mds-3-client', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'osd}', 'overrides/{ignorelist_health', 'tasks/cephfs_misc_tests}'] 2022-06-22T13:03:53.456 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1-mds-3-client', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'tasks/cephfs_misc_tests}'] 2022-06-22T13:03:53.457 INFO:scrape: 2022-06-22T13:03:53.458 INFO:scrape:Failure: Test failure: test_dump_inmemory_log_on_client_eviction (tasks.cephfs.test_misc.TestMisc) 2022-06-22T13:03:53.458 INFO:scrape:1 jobs: ['6892122'] 2022-06-22T13:03:53.459 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1-mds-3-client', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/cephfs_misc_tests}'] 2022-06-22T13:03:53.459 INFO:scrape: 2022-06-22T13:03:53.460 INFO:scrape:Failure: "2022-06-22T12:21:31.239307+0000 mon.a (mon.0) 313 : cluster [WRN] Health check failed: Degraded data redundancy: 2 pgs degraded (PG_DEGRADED)" in cluster log 2022-06-22T13:03:53.460 INFO:scrape:1 jobs: ['6892118'] 2022-06-22T13:03:53.460 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1-mds-2-client', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/cephfs_misc_tests}'] 2022-06-22T13:03:53.461 INFO:scrape: 2022-06-22T13:03:53.462 INFO:teuthology.results:Sending results to jcollin@redhat.com: Test Run: jcollin-2022-06-22_12:04:19-fs-wip-F40633-dump-recent-extraordinary-events-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/jcollin-2022-06-22_12:04:19-fs-wip-F40633-dump-recent-extraordinary-events-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/jcollin-2022-06-22_12:04:19-fs-wip-F40633-dump-recent-extraordinary-events-distro-default-smithi/ failed: 4 dead: 0 running: 0 waiting: 0 queued: 0 passed: 1 Fail ================================================================= [6892118] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:51:40 info: http://pulpito.ceph.com/jcollin-2022-06-22_12:04:19-fs-wip-F40633-dump-recent-extraordinary-events-distro-default-smithi/6892118/ log: http://qa-proxy.ceph.com/teuthology/jcollin-2022-06-22_12:04:19-fs-wip-F40633-dump-recent-extraordinary-events-distro-default-smithi/6892118/ "2022-06-22T12:21:31.239307+0000 mon.a (mon.0) 313 : cluster [WRN] Health check failed: Degraded data redundancy: 2 pgs degraded (PG_DEGRADED)" in cluster log [6892119] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:20:21 info: http://pulpito.ceph.com/jcollin-2022-06-22_12:04:19-fs-wip-F40633-dump-recent-extraordinary-events-distro-default-smithi/6892119/ log: http://qa-proxy.ceph.com/teuthology/jcollin-2022-06-22_12:04:19-fs-wip-F40633-dump-recent-extraordinary-events-distro-default-smithi/6892119/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8c53e1d430104e9aaa3dc536366f6b4a Test failure: test_dump_inmemory_log_on_missed_internal_heartbeats (tasks.cephfs.test_misc.TestMisc) [6892121] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:21:46 info: http://pulpito.ceph.com/jcollin-2022-06-22_12:04:19-fs-wip-F40633-dump-recent-extraordinary-events-distro-default-smithi/6892121/ log: http://qa-proxy.ceph.com/teuthology/jcollin-2022-06-22_12:04:19-fs-wip-F40633-dump-recent-extraordinary-events-distro-default-smithi/6892121/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a13cf5d8e6be4876b9730db5db0a3310 Test failure: test_dump_inmemory_log_on_missed_internal_heartbeats (tasks.cephfs.test_misc.TestMisc) [6892122] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:17:26 info: http://pulpito.ceph.com/jcollin-2022-06-22_12:04:19-fs-wip-F40633-dump-recent-extraordinary-events-distro-default-smithi/6892122/ log: http://qa-proxy.ceph.com/teuthology/jcollin-2022-06-22_12:04:19-fs-wip-F40633-dump-recent-extraordinary-events-distro-default-smithi/6892122/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fd39405e76dd4d21b0bcc7d6b4a598f7 Test failure: test_dump_inmemory_log_on_client_eviction (tasks.cephfs.test_misc.TestMisc) Pass ================================================================= [6892120] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:52:20 info: http://pulpito.ceph.com/jcollin-2022-06-22_12:04:19-fs-wip-F40633-dump-recent-extraordinary-events-distro-default-smithi/6892120/