2023-07-27T02:35:29.780 INFO:root:teuthology version: 0.0.1.dev160+g407880c 2023-07-27T02:35:29.801 INFO:teuthology.results:seed: '4844' 2023-07-27T02:52:48.747 INFO:root:teuthology version: 0.0.1.dev160+g407880c 2023-07-27T02:52:48.750 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-07-27T03:38:52.408 INFO:teuthology.results:Tests finished! gathering results... 2023-07-27T03:38:59.354 INFO:scrape:Found 30 jobs 2023-07-27T03:38:59.355 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354197/teuthology.log 2023-07-27T03:38:59.356 INFO:scrape:Found 1 distinct failure reasons 2023-07-27T03:38:59.356 INFO:scrape:Dead: None 2023-07-27T03:38:59.356 INFO:scrape:3 jobs: ['7354197', '7354211', '7354201'] 2023-07-27T03:38:59.357 INFO:scrape: 2023-07-27T03:38:59.357 INFO:teuthology.results:Sending results to jcollin@redhat.com: Test Run: jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/ failed: 0 dead: 3 running: 0 waiting: 0 queued: 0 passed: 27 Dead ================================================================= [7354197] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354197/ log: http://qa-proxy.ceph.com/teuthology/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354197/ Error reimaging machines: Failed to power on smithi186 [7354201] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354201/ log: http://qa-proxy.ceph.com/teuthology/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354201/ Error reimaging machines: Failed to power on smithi096 [7354211] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354211/ log: http://qa-proxy.ceph.com/teuthology/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354211/ Error reimaging machines: Failed to power on smithi157 Pass ================================================================= [7354182] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:31:39 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354182/ [7354183] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:24:49 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354183/ [7354184] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:26:16 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354184/ [7354185] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:24:27 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354185/ [7354186] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:26:26 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354186/ [7354187] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:31:05 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354187/ [7354188] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:26:06 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354188/ [7354189] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:26:28 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354189/ [7354190] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:24:42 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354190/ [7354191] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:25:12 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354191/ [7354192] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:24:41 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354192/ [7354193] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:30:34 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354193/ [7354194] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:24:27 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354194/ [7354195] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:23:21 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354195/ [7354196] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:26:43 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354196/ [7354198] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:29:27 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354198/ [7354199] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:24:50 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354199/ [7354200] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:25:58 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354200/ [7354202] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:32:53 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354202/ [7354203] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:26:24 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354203/ [7354204] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:29:51 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354204/ [7354205] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:27:09 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354205/ [7354206] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:30:38 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354206/ [7354207] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:27:36 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354207/ [7354208] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:26:39 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354208/ [7354209] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:34:10 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354209/ [7354210] fs:mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{centos_8} workloads/cephfs-mirror-ha-workunit} time: 00:31:33 info: http://pulpito.ceph.com/jcollin-2023-07-27_02:32:18-fs:mirror-ha-wip-cephfs-mirror-fix_quincy-distro-default-smithi/7354210/