2024-02-02T13:08:11.523 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-02-02T13:08:11.523 INFO:teuthology.results:seed: '2068' 2024-02-02T14:38:15.929 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-02-02T14:38:15.966 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-02-02T16:29:24.452 INFO:teuthology.results:Tests finished! gathering results... 2024-02-02T16:29:26.235 INFO:scrape:Found 25 jobs 2024-02-02T16:29:28.771 INFO:scrape:Found 2 distinct failure reasons 2024-02-02T16:29:28.771 INFO:scrape:Failure: Test failure: test_cephfs_mirror_sync_with_purged_snapshot (tasks.cephfs.test_mirroring.TestMirroring) 2024-02-02T16:29:28.771 INFO:scrape:1 jobs: ['7543463'] 2024-02-02T16:29:28.771 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'cephfs-mirror/one-per-cluster', 'clients/{mirror}', 'cluster/{1-node}', 'fs:mirror/{begin/{0-install', 'mount/fuse', 'objectstore/bluestore-bitmap', 'overrides/{ignorelist_health}', 'supported-random-distros$/{centos_latest}', 'tasks/mirror}'] 2024-02-02T16:29:28.772 INFO:scrape: 2024-02-02T16:29:28.772 INFO:scrape:Failure: Test failure: test_mirroring_init_failure_with_recovery (tasks.cephfs.test_mirroring.TestMirroring) 2024-02-02T16:29:28.772 INFO:scrape:1 jobs: ['7543451'] 2024-02-02T16:29:28.772 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'cephfs-mirror/one-per-cluster', 'clients/{mirror}', 'cluster/{1-node}', 'fs:mirror/{begin/{0-install', 'mount/fuse', 'objectstore/bluestore-bitmap', 'overrides/{ignorelist_health}', 'supported-random-distros$/{centos_latest}', 'tasks/mirror}'] 2024-02-02T16:29:28.772 INFO:scrape: 2024-02-02T16:29:28.773 INFO:teuthology.results:Sending results to jcollin@redhat.com: Test Run: jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/ failed: 2 dead: 0 running: 0 waiting: 0 queued: 0 passed: 23 Fail ================================================================= [7543451] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} ----------------------------------------------------------------- time: 01:43:01 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543451/ log: http://qa-proxy.ceph.com/teuthology/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543451/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dc63de039f1a407d8092b52a28a4049e Test failure: test_mirroring_init_failure_with_recovery (tasks.cephfs.test_mirroring.TestMirroring) [7543463] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} ----------------------------------------------------------------- time: 01:26:46 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543463/ log: http://qa-proxy.ceph.com/teuthology/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543463/ sentry: https://sentry.ceph.com/organizations/ceph/?query=47083d1e151f475494975c533231657a Test failure: test_cephfs_mirror_sync_with_purged_snapshot (tasks.cephfs.test_mirroring.TestMirroring) Pass ================================================================= [7543440] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:43 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543440/ [7543441] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:24 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543441/ [7543442] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:27 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543442/ [7543443] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:39 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543443/ [7543444] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:19 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543444/ [7543445] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:35 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543445/ [7543446] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:57 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543446/ [7543447] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:43:06 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543447/ [7543448] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:31 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543448/ [7543449] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:23 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543449/ [7543450] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:34 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543450/ [7543452] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:44 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543452/ [7543453] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:26 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543453/ [7543454] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:25 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543454/ [7543455] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:34 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543455/ [7543456] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:32 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543456/ [7543457] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:27 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543457/ [7543458] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:11 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543458/ [7543459] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:49 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543459/ [7543460] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:13 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543460/ [7543461] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:43:05 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543461/ [7543462] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:43:07 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543462/ [7543464] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:42:39 info: https://pulpito.ceph.com/jcollin-2024-02-02_13:07:10-fs:mirror-wip-jcollin-testing_25Jan2024-distro-default-smithi/7543464/