2020-02-17T16:45:33.107 INFO:root:teuthology version: 1.0.0-7f4dc17 2020-02-17T16:45:33.108 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.79LjZY.tmp -- --verbose --lock --description rados:monthrash/{ceph.yaml clusters/3-mons.yaml mon_election/connectivity.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/many.yaml workloads/rados_mon_workunits.yaml} --name gregf-2020-02-17_05:30:48-rados:monthrash-wip-elector-squash-212-distro-basic-mira --block --owner scheduled_gregf@teuthology --archive /home/teuthworker/archive/gregf-2020-02-17_05:30:48-rados:monthrash-wip-elector-squash-212-distro-basic-mira/4774533 2020-02-17T16:45:33.157 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-02-17T16:45:33.227 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gregf-2020-02-17_05:30:48-rados:monthrash-wip-elector-squash-212-distro-basic-mira/4774533 branch: wip-elector-squash-212 description: rados:monthrash/{ceph.yaml clusters/3-mons.yaml mon_election/connectivity.yaml msgr-failures/few.yaml msgr/async-v1only.yaml objectstore/bluestore-stupid.yaml rados.yaml supported-random-distro$/{rhel_8.yaml} thrashers/many.yaml workloads/rados_mon_workunits.yaml} email: gfarnum@redhat.com first_in_suite: false job_id: '4774533' kernel: kdb: true sha1: distro last_in_suite: false machine_type: mira name: gregf-2020-02-17_05:30:48-rados:monthrash-wip-elector-squash-212-distro-basic-mira nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: rhel os_version: '8.1' overrides: admin_socket: branch: wip-elector-squash-212 ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr2: false ms inject socket failures: 5000 ms type: async mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 25 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 5 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 25 debug rocksdb: 10 mon client ping interval: 4 mon client ping timeout: 12 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(MGR_DOWN\) - \(PG_AVAILABILITY\) - \(SLOW_OPS\) - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - but it is still running - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(AUTH_BAD_CAPS\) mon_bind_msgr2: false sha1: 4e96f6d01c87d23d720091cfdaa0900b92d41fab ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: sha1: 4e96f6d01c87d23d720091cfdaa0900b92d41fab thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-elector-qa sha1: 620894dc894e35d055c06b84b6a83bb6db81b626 owner: scheduled_gregf@teuthology priority: 101 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - client.0 sha1: 4e96f6d01c87d23d720091cfdaa0900b92d41fab suite: rados:monthrash suite_branch: wip-elector-qa suite_path: /home/teuthworker/src/github.com_gregsfortytwo_ceph_wip-elector-qa/qa suite_relpath: qa suite_repo: https://github.com/gregsfortytwo/ceph.git suite_sha1: 620894dc894e35d055c06b84b6a83bb6db81b626 tasks: - install: null - ceph: null - mon_thrash: freeze_mon_duration: 20 freeze_mon_probability: 10 revive_delay: 20 thrash_delay: 1 thrash_many: true - workunit: clients: client.0: - mon/pool_ops.sh - mon/crush_ops.sh - mon/osd.sh - mon/caps.sh teuthology_branch: master tube: mira verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.mira.10671 2020-02-17T16:45:33.228 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_gregsfortytwo_ceph_wip-elector-qa/qa; will attempt to use it 2020-02-17T16:45:33.230 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_gregsfortytwo_ceph_wip-elector-qa/qa/tasks 2020-02-17T16:45:33.230 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-02-17T16:45:33.232 INFO:teuthology.task.internal:Checking packages... 2020-02-17T16:45:33.266 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'basic' and ceph hash '4e96f6d01c87d23d720091cfdaa0900b92d41fab' 2020-02-17T16:45:33.267 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-02-17T16:45:33.267 INFO:teuthology.packaging:ref: None 2020-02-17T16:45:33.267 INFO:teuthology.packaging:tag: None 2020-02-17T16:45:33.267 INFO:teuthology.packaging:branch: wip-elector-squash-212 2020-02-17T16:45:33.267 INFO:teuthology.packaging:sha1: 4e96f6d01c87d23d720091cfdaa0900b92d41fab 2020-02-17T16:45:33.267 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-elector-squash-212 2020-02-17T16:45:33.433 INFO:teuthology.task.internal:Found packages for ceph version 15.1.0-348.g4e96f6d 2020-02-17T16:45:33.434 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-02-17T16:45:33.465 INFO:teuthology.task.internal:no buildpackages task found 2020-02-17T16:45:33.465 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-02-17T16:45:33.518 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-02-17T16:45:33.518 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-02-17T16:45:33.582 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:45:43.608 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:45:53.648 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:46:03.669 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:46:13.687 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:46:23.706 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:46:33.723 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:46:43.740 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:46:53.757 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:47:03.773 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:47:13.795 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:47:23.813 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:47:33.832 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:47:43.852 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:47:53.872 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:48:03.892 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:48:13.910 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:48:23.927 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:48:33.938 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:48:43.957 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:48:53.970 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:49:03.987 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:49:14.005 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:49:24.023 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:49:34.038 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:49:44.055 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:49:54.071 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:50:04.088 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:50:14.102 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:50:24.122 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:50:34.142 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:50:44.159 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:50:54.174 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:51:04.194 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:51:14.211 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:51:24.230 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:51:34.250 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:51:44.268 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:51:54.287 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:52:04.304 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:52:14.320 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:52:24.341 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:52:34.360 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:52:44.380 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:52:54.396 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:53:04.419 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:53:14.435 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:53:24.451 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:53:34.469 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:53:44.487 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:53:54.502 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:54:04.519 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:54:14.534 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:54:24.556 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:54:34.570 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:54:44.588 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:54:54.603 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:55:04.621 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:55:14.638 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:55:24.656 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:55:34.675 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:55:44.686 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:55:54.703 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:56:04.722 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:56:14.740 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:56:24.757 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:56:34.770 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:56:44.787 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:56:54.805 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:57:04.823 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:57:14.838 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:57:24.856 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:57:34.873 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:57:44.894 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:57:54.911 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:58:04.929 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:58:14.946 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:58:24.962 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:58:34.980 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:58:44.998 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:58:55.011 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:59:05.030 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:59:15.042 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:59:25.064 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:59:35.082 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:59:45.099 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T16:59:55.116 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:00:05.125 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:00:15.143 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:00:25.160 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:00:35.179 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:00:45.196 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:00:55.206 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:01:05.224 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:01:15.244 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:01:25.259 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:01:35.275 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:01:45.294 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:01:55.311 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:02:05.327 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:02:15.346 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:02:25.363 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:02:35.382 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:02:45.401 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:02:55.421 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:03:05.439 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:03:15.457 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:03:25.478 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:03:35.495 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:03:45.525 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:03:55.543 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:04:05.560 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:04:15.578 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:04:25.595 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:04:35.612 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:04:45.633 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:04:55.652 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:05:05.668 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:05:15.683 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:05:25.702 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:05:35.721 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:05:45.739 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:05:55.755 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:06:05.771 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:06:15.787 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:06:25.804 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:06:35.818 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:06:45.835 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:06:55.854 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:07:05.871 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:07:15.899 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:07:25.916 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:07:35.934 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:07:45.951 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:07:55.968 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:08:05.980 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:08:15.996 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:08:26.015 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:08:36.038 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:08:46.055 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:08:56.073 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:09:06.090 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 2)... 2020-02-17T17:09:16.108 INFO:teuthology.task.internal.lock_machines:waiting for more mira machines to be free (need 2 + 2, have 3)... 2020-02-17T17:09:26.121 DEBUG:teuthology.lock.ops:lock_many request: {'count': 2, 'locked_by': 'scheduled_gregf@teuthology', 'description': '/home/teuthworker/archive/gregf-2020-02-17_05:30:48-rados:monthrash-wip-elector-squash-212-distro-basic-mira/4774533', 'machine_type': 'mira'} 2020-02-17T17:09:26.137 DEBUG:teuthology.lock.ops:locked ubuntu@mira110.front.sepia.ceph.com, ubuntu@mira101.front.sepia.ceph.com 2020-02-17T17:09:26.191 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2020-02-17T17:09:26.397 INFO:teuthology.provision.fog.mira101:Scheduling deploy of rhel 8.1 2020-02-17T17:09:26.426 INFO:teuthology.provision.fog.mira110:Scheduling deploy of rhel 8.1 2020-02-17T17:09:26.910 INFO:teuthology.orchestra.console:Power off mira110 2020-02-17T17:09:26.910 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-02-17T17:09:26.935 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:26.955 INFO:teuthology.orchestra.console:Power off mira101 2020-02-17T17:09:26.956 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-02-17T17:09:26.985 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:27.069 INFO:teuthology.orchestra.console:Power off for mira110 completed 2020-02-17T17:09:27.113 INFO:teuthology.orchestra.console:Power off for mira101 completed 2020-02-17T17:09:27.170 INFO:teuthology.orchestra.console:Power on mira110 2020-02-17T17:09:27.170 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-02-17T17:09:27.195 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:27.214 INFO:teuthology.orchestra.console:Power on mira101 2020-02-17T17:09:27.214 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-02-17T17:09:27.243 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:31.227 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:31.272 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:35.342 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:35.385 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-02-17T17:09:35.552 INFO:teuthology.orchestra.console:Power on for mira110 completed 2020-02-17T17:09:35.597 INFO:teuthology.orchestra.console:Power on for mira101 completed 2020-02-17T17:09:35.654 INFO:teuthology.provision.fog.mira110:Waiting for deploy to finish 2020-02-17T17:09:35.698 INFO:teuthology.provision.fog.mira101:Waiting for deploy to finish 2020-02-17T17:12:24.474 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2020-02-17T17:12:24.474 DEBUG:teuthology.task.console_log:Killing console logger for mira110 2020-02-17T17:12:24.475 DEBUG:teuthology.task.console_log:Killing console logger for mira101 2020-02-17T17:12:24.475 DEBUG:teuthology.exit:Finished running handlers