2024-03-06T13:59:23.146 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-03-06T13:59:23.146 INFO:teuthology.results:seed: '7868' 2024-03-06T14:04:24.665 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-03-06T14:04:24.668 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-03-06T14:52:28.377 INFO:teuthology.results:Tests finished! gathering results... 2024-03-06T14:52:28.809 INFO:scrape:Found 12 jobs 2024-03-06T14:52:40.396 INFO:scrape:Found 2 distinct failure reasons 2024-03-06T14:52:40.397 INFO:scrape:Failure: Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) 2024-03-06T14:52:40.397 INFO:scrape:8 jobs: ['7583485', '7583490', '7583487', '7583484', '7583488', '7583482', '7583489', '7583493'] 2024-03-06T14:52:40.397 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'osd}', 'overrides/{ignorelist_health', 'tasks/quiesce}'] 2024-03-06T14:52:40.397 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_latest}', 'distro/{ubuntu_latest}', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'k-stock}', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{centos_9.stream', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'subvol_versions/create_subvol_version_v1', 'subvol_versions/create_subvol_version_v2', 'tasks/quiesce}'] 2024-03-06T14:52:40.397 INFO:scrape: 2024-03-06T14:52:40.397 INFO:scrape:Failure: Test failure: test_quiesce_path_kill (tasks.cephfs.test_quiesce.TestQuiesce) 2024-03-06T14:52:40.397 INFO:scrape:1 jobs: ['7583486'] 2024-03-06T14:52:40.398 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'subvol_versions/create_subvol_version_v2', 'tasks/quiesce}'] 2024-03-06T14:52:40.398 INFO:scrape: 2024-03-06T14:52:40.399 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/ failed: 9 dead: 0 running: 0 waiting: 0 queued: 0 passed: 3 Fail ================================================================= [7583482] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/quiesce} ----------------------------------------------------------------- time: 00:24:03 info: https://pulpito.ceph.com/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583482/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583482/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9957408d3d7342dd970f0f355af4ca59 Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) [7583484] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/quiesce} ----------------------------------------------------------------- time: 00:19:22 info: https://pulpito.ceph.com/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583484/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583484/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7798753b948445af81c2d37a1b7f33b5 Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) [7583485] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/quiesce} ----------------------------------------------------------------- time: 00:23:36 info: https://pulpito.ceph.com/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583485/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583485/ sentry: https://sentry.ceph.com/organizations/ceph/?query=957c79cd497143f6981f72d76f1ca2b8 Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) [7583486] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/quiesce} ----------------------------------------------------------------- time: 00:18:57 info: https://pulpito.ceph.com/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583486/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583486/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2fed228f8c714f9eb955a39e38527133 Test failure: test_quiesce_path_kill (tasks.cephfs.test_quiesce.TestQuiesce) [7583487] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/quiesce} ----------------------------------------------------------------- time: 00:20:25 info: https://pulpito.ceph.com/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583487/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583487/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a1dc2833b2094431963f89d08b7a10c0 Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) [7583488] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/quiesce} ----------------------------------------------------------------- time: 00:23:28 info: https://pulpito.ceph.com/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583488/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583488/ sentry: https://sentry.ceph.com/organizations/ceph/?query=56ed48dec48f426799c9a9c4d0eee3c4 Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) [7583489] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/quiesce} ----------------------------------------------------------------- time: 00:21:10 info: https://pulpito.ceph.com/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583489/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583489/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0ba133e048d4446eb1c3853ca786b82d Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) [7583490] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/quiesce} ----------------------------------------------------------------- time: 00:19:00 info: https://pulpito.ceph.com/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583490/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583490/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2e13e39e4ceb4fe984c0a38a1fdc018b Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) [7583493] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/quiesce} ----------------------------------------------------------------- time: 00:19:03 info: https://pulpito.ceph.com/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583493/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583493/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d00cfa903e654f52a0279aee7ab41e37 Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) Pass ================================================================= [7583483] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/quiesce} time: 00:33:44 info: https://pulpito.ceph.com/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583483/ [7583491] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/quiesce} time: 00:39:15 info: https://pulpito.ceph.com/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583491/ [7583492] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/quiesce} time: 00:33:18 info: https://pulpito.ceph.com/pdonnell-2024-03-06_13:32:55-fs:functional-wip-batrick-testing-20240305.201857-distro-default-smithi/7583492/