2024-03-26T01:06:46.507 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-03-26T01:06:46.507 INFO:teuthology.results:seed: '7125' 2024-03-26T01:14:21.317 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-03-26T01:14:21.321 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-03-26T03:22:31.355 INFO:teuthology.results:Tests finished! gathering results... 2024-03-26T03:22:32.505 INFO:scrape:Found 25 jobs 2024-03-26T03:22:35.003 INFO:scrape:Found 2 distinct failure reasons 2024-03-26T03:22:35.003 INFO:scrape:Failure: Test failure: test_mirroring_init_failure_with_recovery (tasks.cephfs.test_mirroring.TestMirroring) 2024-03-26T03:22:35.004 INFO:scrape:1 jobs: ['7622338'] 2024-03-26T03:22:35.004 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'cephfs-mirror/one-per-cluster', 'clients/{mirror}', 'cluster/{1-node}', 'fs:mirror/{begin/{0-install', 'mount/fuse', 'objectstore/bluestore-bitmap', 'overrides/{ignorelist_health}', 'supported-random-distros$/{centos_latest}', 'tasks/mirror}'] 2024-03-26T03:22:35.004 INFO:scrape: 2024-03-26T03:22:35.004 INFO:scrape:Failure: Test failure: test_cephfs_mirror_cancel_mirroring_and_readd (tasks.cephfs.test_mirroring.TestMirroring) 2024-03-26T03:22:35.004 INFO:scrape:1 jobs: ['7622345'] 2024-03-26T03:22:35.004 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'cephfs-mirror/one-per-cluster', 'clients/{mirror}', 'cluster/{1-node}', 'fs:mirror/{begin/{0-install', 'mount/fuse', 'objectstore/bluestore-bitmap', 'overrides/{ignorelist_health}', 'supported-random-distros$/{centos_latest}', 'tasks/mirror}'] 2024-03-26T03:22:35.004 INFO:scrape: 2024-03-26T03:22:35.005 INFO:teuthology.results:Sending results to jcollin@redhat.com: Test Run: jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/ failed: 2 dead: 0 running: 0 waiting: 0 queued: 0 passed: 23 Fail ================================================================= [7622338] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} ----------------------------------------------------------------- time: 01:59:07 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622338/ log: http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622338/ sentry: https://sentry.ceph.com/organizations/ceph/?query=36772eb44f3b434f82307f5e87192efd Test failure: test_mirroring_init_failure_with_recovery (tasks.cephfs.test_mirroring.TestMirroring) [7622345] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} ----------------------------------------------------------------- time: 00:22:37 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622345/ log: http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622345/ sentry: https://sentry.ceph.com/organizations/ceph/?query=108cfb28225e4fdc8da2932d81a427b1 Test failure: test_cephfs_mirror_cancel_mirroring_and_readd (tasks.cephfs.test_mirroring.TestMirroring) Pass ================================================================= [7622329] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:22 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622329/ [7622330] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:02 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622330/ [7622331] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 02:00:45 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622331/ [7622332] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:57:55 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622332/ [7622333] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:59 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622333/ [7622334] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:10 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622334/ [7622335] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:27 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622335/ [7622336] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:29 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622336/ [7622337] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:40 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622337/ [7622339] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:40 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622339/ [7622340] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:52 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622340/ [7622341] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:28 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622341/ [7622342] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:47 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622342/ [7622343] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:26 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622343/ [7622344] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:01 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622344/ [7622346] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:57:41 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622346/ [7622347] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:59:01 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622347/ [7622348] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:57:54 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622348/ [7622349] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:31 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622349/ [7622350] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:13 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622350/ [7622351] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:27 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622351/ [7622352] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:58:58 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622352/ [7622353] fs:mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_latest} tasks/mirror} time: 01:57:48 info: https://pulpito.ceph.com/jcollin-2024-03-26_01:04:24-fs:mirror-wip-jcollin-testing-25032024-distro-default-smithi/7622353/