2020-02-17T16:49:41.280 INFO:root:teuthology version: 1.0.0-7f4dc17 2020-02-17T16:49:41.281 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.9UqJ6F.tmp -- --verbose --lock --description rados:monthrash/{ceph.yaml clusters/3-mons.yaml mon_election/connectivity.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/sync-many.yaml workloads/pool-create-delete.yaml} --name gregf-2020-02-17_05:30:48-rados:monthrash-wip-elector-squash-212-distro-basic-mira --block --owner scheduled_gregf@teuthology --archive /home/teuthworker/archive/gregf-2020-02-17_05:30:48-rados:monthrash-wip-elector-squash-212-distro-basic-mira/4774535 2020-02-17T16:49:41.316 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-02-17T16:49:41.389 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gregf-2020-02-17_05:30:48-rados:monthrash-wip-elector-squash-212-distro-basic-mira/4774535 branch: wip-elector-squash-212 description: rados:monthrash/{ceph.yaml clusters/3-mons.yaml mon_election/connectivity.yaml msgr-failures/few.yaml msgr/async.yaml objectstore/filestore-xfs.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/sync-many.yaml workloads/pool-create-delete.yaml} email: gfarnum@redhat.com first_in_suite: false job_id: '4774535' kernel: kdb: true sha1: distro last_in_suite: false machine_type: mira name: gregf-2020-02-17_05:30:48-rados:monthrash-wip-elector-squash-212-distro-basic-mira nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: rhel os_version: '8.1' overrides: admin_socket: branch: wip-elector-squash-212 ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms inject socket failures: 5000 ms type: async mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 25 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos min: 10 paxos service trim min: 5 paxos trim min: 10 osd: debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 25 osd debug verify cached snaps: true osd debug verify missing on start: true osd objectstore: filestore osd op queue: debug_random osd op queue cut off: debug_random osd sloppy crc: true fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(MGR_DOWN\) - \(PG_AVAILABILITY\) - \(SLOW_OPS\) - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - slow request - overall HEALTH_ - \(POOL_APP_NOT_ENABLED\) sha1: 4e96f6d01c87d23d720091cfdaa0900b92d41fab ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: osd objectstore: filestore osd sloppy crc: true filestore: true fs: xfs install: ceph: sha1: 4e96f6d01c87d23d720091cfdaa0900b92d41fab workunit: branch: wip-elector-qa sha1: 620894dc894e35d055c06b84b6a83bb6db81b626 owner: scheduled_gregf@teuthology priority: 101 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - client.0 sha1: 4e96f6d01c87d23d720091cfdaa0900b92d41fab suite: rados:monthrash suite_branch: wip-elector-qa suite_path: /home/teuthworker/src/github.com_gregsfortytwo_ceph_wip-elector-qa/qa suite_relpath: qa suite_repo: https://github.com/gregsfortytwo/ceph.git suite_sha1: 620894dc894e35d055c06b84b6a83bb6db81b626 tasks: - install: null - ceph: null - mon_thrash: revive_delay: 90 thrash_delay: 1 thrash_many: true - exec: client.0: - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel teuthology_branch: master tube: mira verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.mira.10673 2020-02-17T16:49:41.389 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_gregsfortytwo_ceph_wip-elector-qa/qa; will attempt to use it 2020-02-17T16:49:41.390 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_gregsfortytwo_ceph_wip-elector-qa/qa/tasks 2020-02-17T16:49:41.390 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-02-17T16:49:41.392 INFO:teuthology.task.internal:Checking packages... 2020-02-17T16:49:41.405 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'basic' and ceph hash '4e96f6d01c87d23d720091cfdaa0900b92d41fab' 2020-02-17T16:49:41.405 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-02-17T16:49:41.405 INFO:teuthology.packaging:ref: None 2020-02-17T16:49:41.405 INFO:teuthology.packaging:tag: None 2020-02-17T16:49:41.405 INFO:teuthology.packaging:branch: wip-elector-squash-212 2020-02-17T16:49:41.406 INFO:teuthology.packaging:sha1: 4e96f6d01c87d23d720091cfdaa0900b92d41fab 2020-02-17T16:49:41.406 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-elector-squash-212 2020-02-17T16:49:41.562 INFO:teuthology.task.internal:Found packages for ceph version 15.1.0-348.g4e96f6d 2020-02-17T16:49:41.563 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-02-17T16:49:41.574 INFO:teuthology.task.internal:no buildpackages task found 2020-02-17T16:49:41.574 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-02-17T16:49:41.587 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-02-17T16:49:41.587 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-02-17T16:49:41.634 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:49:51.652 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:50:01.674 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:50:11.691 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:50:21.703 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:50:31.720 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:50:41.741 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:50:51.755 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:51:01.773 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:51:11.791 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:51:21.809 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:51:31.828 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:51:41.846 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:51:51.866 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:52:01.886 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:52:11.905 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:52:21.924 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:52:31.943 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:52:41.959 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:52:51.975 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:53:01.991 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:53:12.007 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:53:22.018 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:53:32.036 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:53:42.054 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:53:52.072 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:54:02.089 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:54:12.108 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:54:22.127 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:54:32.143 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:54:42.162 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:54:52.181 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:55:02.199 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:55:12.215 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:55:22.231 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:55:32.249 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:55:42.267 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:55:52.284 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:56:02.299 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:56:12.317 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:56:22.335 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:56:32.351 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:56:42.367 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:56:52.384 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:57:02.399 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:57:12.414 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:57:22.430 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:57:32.448 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:57:42.466 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:57:52.483 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:58:02.499 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:58:12.516 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:58:22.531 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:58:32.547 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:58:42.562 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:58:52.580 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:59:02.599 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:59:12.617 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:59:22.636 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:59:32.652 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:59:42.679 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:59:52.696 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:00:02.714 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:00:12.724 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:00:22.739 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:00:32.756 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:00:42.775 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:00:52.790 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:01:02.807 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:01:12.822 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:01:22.839 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:01:32.857 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:01:42.875 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:01:52.891 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:02:02.908 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:02:12.923 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:02:22.939 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:02:32.949 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:02:42.968 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:02:52.984 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:03:03.002 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:03:13.018 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:03:23.035 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:03:33.054 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:03:43.072 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:03:53.090 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:04:03.108 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:04:13.124 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:04:23.141 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:04:33.160 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:04:43.178 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:04:53.195 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:05:03.213 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:05:13.231 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:05:23.247 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:05:33.266 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:05:43.277 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:05:53.295 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:06:03.313 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:06:13.331 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:06:23.349 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:06:33.369 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:06:43.387 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:06:53.403 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:07:03.420 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:07:13.435 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:07:23.451 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:07:33.466 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:07:43.489 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:07:53.508 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:08:03.528 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:08:13.543 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:08:23.559 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:08:33.577 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:08:43.592 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:08:53.612 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:09:03.632 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:09:13.651 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 3)... 2020-02-17T17:09:23.667 DEBUG:teuthology.lock.ops:lock_many request: {'count': 2, 'locked_by': 'scheduled_gregf@teuthology', 'description': '/home/teuthworker/archive/gregf-2020-02-17_05:30:48-rados:monthrash-wip-elector-squash-212-distro-basic-mira/4774535', 'machine_type': 'mira'} 2020-02-17T17:09:23.685 DEBUG:teuthology.lock.ops:locked ubuntu@mira109.front.sepia.ceph.com, ubuntu@mira062.front.sepia.ceph.com 2020-02-17T17:09:23.744 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2020-02-17T17:09:24.009 INFO:teuthology.provision.fog.mira109:Scheduling deploy of rhel 8.1 2020-02-17T17:09:24.067 INFO:teuthology.provision.fog.mira062:Scheduling deploy of rhel 8.1 2020-02-17T17:09:24.481 INFO:teuthology.orchestra.console:Power off mira109 2020-02-17T17:09:24.481 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira109.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-02-17T17:09:24.505 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira109.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:24.612 INFO:teuthology.orchestra.console:Power off mira062 2020-02-17T17:09:24.613 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-02-17T17:09:24.631 INFO:teuthology.orchestra.console:Power off for mira109 completed 2020-02-17T17:09:24.638 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:24.733 INFO:teuthology.orchestra.console:Power on mira109 2020-02-17T17:09:24.733 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira109.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-02-17T17:09:24.759 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira109.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:24.771 INFO:teuthology.orchestra.console:Power off for mira062 completed 2020-02-17T17:09:24.872 INFO:teuthology.orchestra.console:Power on mira062 2020-02-17T17:09:24.872 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-02-17T17:09:24.895 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:28.791 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira109.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:28.922 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:32.902 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira109.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:33.034 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:33.114 INFO:teuthology.orchestra.console:Power on for mira109 completed 2020-02-17T17:09:33.215 INFO:teuthology.provision.fog.mira109:Waiting for deploy to finish 2020-02-17T17:09:33.245 INFO:teuthology.orchestra.console:Power on for mira062 completed 2020-02-17T17:09:33.346 INFO:teuthology.provision.fog.mira062:Waiting for deploy to finish 2020-02-17T17:12:24.492 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2020-02-17T17:12:24.492 DEBUG:teuthology.task.console_log:Killing console logger for mira062 2020-02-17T17:12:24.492 DEBUG:teuthology.task.console_log:Killing console logger for mira109 2020-02-17T17:12:24.493 DEBUG:teuthology.exit:Finished running handlers