2024-02-18T12:30:48.801 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-02-18T12:30:48.802 INFO:teuthology.results:seed: '1719' 2024-02-18T13:14:15.512 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-02-18T13:14:15.545 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-02-18T13:54:18.634 INFO:teuthology.results:Tests finished! gathering results... 2024-02-18T13:54:19.704 INFO:scrape:Found 12 jobs 2024-02-18T13:54:25.875 INFO:scrape:Found 2 distinct failure reasons 2024-02-18T13:54:25.876 INFO:scrape:Failure: Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) 2024-02-18T13:54:25.876 INFO:scrape:9 jobs: ['7565657', '7565651', '7565646', '7565648', '7565652', '7565655', '7565649', '7565654', '7565653'] 2024-02-18T13:54:25.876 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'osd}', 'overrides/{ignorelist_health', 'tasks/quiesce}'] 2024-02-18T13:54:25.876 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_latest}', 'distro/{ubuntu_latest}', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'k-stock}', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{centos_9.stream', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'subvol_versions/create_subvol_version_v1', 'subvol_versions/create_subvol_version_v2', 'tasks/quiesce}'] 2024-02-18T13:54:25.876 INFO:scrape: 2024-02-18T13:54:25.876 INFO:scrape:Failure: Test failure: test_quiesce_path_multirank (tasks.cephfs.test_quiesce.TestQuiesceMultiRank) 2024-02-18T13:54:25.876 INFO:scrape:1 jobs: ['7565656'] 2024-02-18T13:54:25.877 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'subvol_versions/create_subvol_version_v2', 'tasks/quiesce}'] 2024-02-18T13:54:25.877 INFO:scrape: 2024-02-18T13:54:25.877 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/ failed: 10 dead: 0 running: 0 waiting: 0 queued: 0 passed: 2 Fail ================================================================= [7565646] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/quiesce} ----------------------------------------------------------------- time: 00:21:32 info: https://pulpito.ceph.com/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565646/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565646/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a8d5a9a2296a494993f0304250c1a150 Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) [7565648] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/quiesce} ----------------------------------------------------------------- time: 00:16:35 info: https://pulpito.ceph.com/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565648/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565648/ sentry: https://sentry.ceph.com/organizations/ceph/?query=246f4b69f6bd471584648a4ee51a08bf Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) [7565649] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/quiesce} ----------------------------------------------------------------- time: 00:21:38 info: https://pulpito.ceph.com/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565649/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565649/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8b35521018a34e36a007fa2d912def3d Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) [7565651] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/quiesce} ----------------------------------------------------------------- time: 00:16:35 info: https://pulpito.ceph.com/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565651/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565651/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2bdfe4890db94004b10b0dac423c0b4b Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) [7565652] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/quiesce} ----------------------------------------------------------------- time: 00:21:03 info: https://pulpito.ceph.com/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565652/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565652/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8a47bd25ffa54f519b0b4d727b6ca2b5 Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) [7565653] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/quiesce} ----------------------------------------------------------------- time: 00:22:56 info: https://pulpito.ceph.com/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565653/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565653/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f46210136c6a488a82fc6d184d11ecec Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) [7565654] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/quiesce} ----------------------------------------------------------------- time: 00:16:12 info: https://pulpito.ceph.com/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565654/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565654/ sentry: https://sentry.ceph.com/organizations/ceph/?query=221824a1f4bd49b8920a71d7465f659e Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) [7565655] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/quiesce} ----------------------------------------------------------------- time: 00:21:29 info: https://pulpito.ceph.com/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565655/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565655/ sentry: https://sentry.ceph.com/organizations/ceph/?query=144ea1b2e807436a868670083b97c58c Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) [7565656] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/quiesce} ----------------------------------------------------------------- time: 00:32:29 info: https://pulpito.ceph.com/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565656/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565656/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b074f23475e34cc0acb289e9e052ef08 Test failure: test_quiesce_path_multirank (tasks.cephfs.test_quiesce.TestQuiesceMultiRank) [7565657] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/quiesce} ----------------------------------------------------------------- time: 00:16:38 info: https://pulpito.ceph.com/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565657/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565657/ sentry: https://sentry.ceph.com/organizations/ceph/?query=35825565075e4bcfb5d99643f30631d1 Test failure: test_quiesce_path_release (tasks.cephfs.test_quiesce.TestQuiesce) Pass ================================================================= [7565647] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/quiesce} time: 00:30:46 info: https://pulpito.ceph.com/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565647/ [7565650] fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/quiesce} time: 00:26:42 info: https://pulpito.ceph.com/pdonnell-2024-02-17_21:40:39-fs:functional-wip-batrick-testing-20240217.153951-distro-default-smithi/7565650/