2020-06-12T02:02:26.173 INFO:root:teuthology version: 1.0.0-6c966fe 2020-06-12T02:02:26.176 DEBUG:teuthology.run:Teuthology command: teuthology --lock --owner scheduled_kchai@teuthology --block --name kchai-2020-06-11_10:13:29-rados-octopus-distro-basic-smithi --verbose --description rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} --archive /home/teuthworker/archive/kchai-2020-06-11_10:13:29-rados-octopus-distro-basic-smithi/5137979 -- /tmp/teuthology-worker.wvax1a_n.tmp 2020-06-12T02:02:26.226 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-06-12T02:02:26.308 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/kchai-2020-06-11_10:13:29-rados-octopus-distro-basic-smithi/5137979 branch: octopus description: rados/thrash-erasure-code-shec/{ceph.yaml clusters/{fixed-4.yaml openstack.yaml} msgr-failures/fastclose.yaml objectstore/bluestore-comp-snappy.yaml rados.yaml recovery-overrides/{more-active-recovery.yaml} supported-random-distro$/{ubuntu_latest.yaml} thrashers/default.yaml thrashosds-health.yaml workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml} email: tchaikov@gmail.com first_in_suite: false job_id: '5137979' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: kchai-2020-06-11_10:13:29-rados-octopus-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: octopus ceph: conf: global: mon client directed command retry: 5 ms inject socket failures: 2500 ms tcp read timeout: 5 osd_recovery_max_active: 10 osd_recovery_max_single_start: 10 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug reject backfill probability: 0.1 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - slow request - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request sha1: 91d24f498556f86f6d1318d2efd5210c35b36762 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: 91d24f498556f86f6d1318d2efd5210c35b36762 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: octopus-py3 sha1: 1e0d5d60071d5554f87a2be72ea0bcccb272aab3 owner: scheduled_kchai@teuthology priority: 90 repo: https://github.com/ceph/ceph.git roles: - - mon.a - mgr.y - osd.0 - osd.4 - osd.8 - osd.12 - - mon.b - osd.1 - osd.5 - osd.9 - osd.13 - - mon.c - osd.2 - osd.6 - osd.10 - osd.14 - - mgr.x - osd.3 - osd.7 - osd.11 - osd.15 - client.0 sha1: 91d24f498556f86f6d1318d2efd5210c35b36762 sleep_before_teardown: 0 suite: rados suite_branch: octopus-py3 suite_path: /home/teuthworker/src/github.com_tchaikov_ceph_octopus-py3/qa suite_relpath: qa suite_repo: https://github.com/tchaikov/ceph.git suite_sha1: 1e0d5d60071d5554f87a2be72ea0bcccb272aab3 tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 min_in: 8 timeout: 1200 - rados: clients: - client.0 ec_pool: true erasure_code_profile: c: 2 crush-failure-domain: osd k: 4 m: 3 name: shecprofile plugin: shec objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 400 write_append_excl: false teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.7008 2020-06-12T02:02:26.309 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_tchaikov_ceph_octopus-py3/qa; will attempt to use it 2020-06-12T02:02:26.310 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_tchaikov_ceph_octopus-py3/qa/tasks 2020-06-12T02:02:26.310 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-06-12T02:02:26.316 INFO:teuthology.task.internal:Checking packages... 2020-06-12T02:02:26.341 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash '91d24f498556f86f6d1318d2efd5210c35b36762' 2020-06-12T02:02:26.341 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-06-12T02:02:26.342 INFO:teuthology.packaging:ref: None 2020-06-12T02:02:26.342 INFO:teuthology.packaging:tag: None 2020-06-12T02:02:26.343 INFO:teuthology.packaging:branch: octopus 2020-06-12T02:02:26.343 INFO:teuthology.packaging:sha1: 91d24f498556f86f6d1318d2efd5210c35b36762 2020-06-12T02:02:26.344 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=octopus 2020-06-12T02:02:26.727 INFO:teuthology.task.internal:Found packages for ceph version 15.2.3-298-g7488806bf1-1bionic 2020-06-12T02:02:26.728 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-06-12T02:02:26.738 INFO:teuthology.task.internal:no buildpackages task found 2020-06-12T02:02:26.738 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-06-12T02:02:26.753 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-06-12T02:02:26.753 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-06-12T02:02:26.827 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:02:36.856 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:02:46.880 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:02:56.916 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:03:07.243 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:03:17.274 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:03:27.292 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:03:37.324 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:03:47.343 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:03:57.369 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-06-12T02:04:07.389 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:04:17.407 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:04:27.431 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:04:37.455 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:04:47.477 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:04:57.497 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:05:07.517 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:05:17.541 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:05:27.564 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-06-12T02:05:37.585 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:05:47.609 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-06-12T02:05:57.636 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:06:07.660 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:06:21.245 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:06:31.271 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:06:41.295 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:06:51.315 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:07:01.335 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:07:11.375 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:07:21.394 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:07:31.412 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:07:41.435 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:07:51.453 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:08:01.472 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-06-12T02:08:11.491 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:08:21.510 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:08:31.528 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:08:41.547 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:08:51.571 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:09:01.588 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:09:11.609 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-06-12T02:09:21.629 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:09:31.661 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:09:41.682 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:09:51.703 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:10:01.719 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:10:11.735 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-06-12T02:10:21.768 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:10:31.791 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:10:41.807 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:10:51.824 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:11:01.845 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:11:11.870 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:11:21.888 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:11:31.908 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:11:41.927 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:11:51.945 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:12:01.970 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:12:12.014 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:12:22.031 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:12:32.051 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:12:42.074 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:12:52.092 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:13:02.108 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-06-12T02:13:12.135 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:13:22.166 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:13:32.183 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:13:42.210 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-06-12T02:13:52.230 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:14:02.252 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:14:12.268 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:14:22.286 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-06-12T02:14:32.307 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:14:42.324 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:14:52.343 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:15:02.363 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:15:12.380 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:15:22.411 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:15:32.429 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:15:42.453 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:15:52.477 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:16:02.511 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:16:12.524 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:16:22.546 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:16:32.564 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:16:42.590 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:16:52.610 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:17:02.629 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:17:12.648 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:17:22.668 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:17:32.683 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:17:42.700 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:17:52.721 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:18:02.740 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:18:12.760 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-06-12T02:18:22.782 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:18:32.801 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:18:42.819 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:18:52.833 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:19:02.854 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:19:12.874 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:19:22.890 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:19:32.907 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:19:42.917 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:19:52.954 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-06-12T02:20:02.973 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:20:13.003 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-06-12T02:20:23.020 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:20:33.046 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:20:43.063 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:20:53.079 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:21:03.098 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:21:13.123 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:21:23.143 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:21:33.159 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:21:43.183 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:21:53.200 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:22:03.220 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:22:13.240 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:22:23.260 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:22:33.282 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:22:43.299 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:22:53.317 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:23:03.337 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:23:13.357 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:23:23.373 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:23:33.396 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:23:43.420 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:23:53.439 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:24:03.458 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:24:13.475 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:24:23.497 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:24:33.519 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:24:43.534 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:24:53.552 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:25:03.572 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:25:13.591 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:25:23.610 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-06-12T02:25:33.637 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:25:43.659 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:25:53.694 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:26:03.719 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:26:13.738 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:26:23.755 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:26:33.772 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:26:43.794 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:26:53.814 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:27:03.847 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:27:13.869 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-06-12T02:27:23.895 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:27:33.919 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:27:43.936 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:27:53.956 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:28:03.985 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:28:14.004 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 3)... 2020-06-12T02:28:24.040 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-06-12T02:28:34.061 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:28:44.085 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:28:54.106 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:29:04.129 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:29:14.148 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:29:24.166 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:29:34.186 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:29:44.209 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:29:54.227 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-06-12T02:30:04.244 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:30:14.265 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:30:24.285 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:30:34.305 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:30:44.325 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:30:54.360 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:31:04.390 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:31:14.416 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:31:24.432 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:31:34.452 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:31:44.471 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:31:54.487 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:32:04.509 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:32:14.522 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:32:24.539 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:32:34.555 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:32:44.571 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:32:54.586 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:33:04.603 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:33:14.635 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:33:24.654 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 2)... 2020-06-12T02:33:34.672 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:33:44.692 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:33:54.712 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:34:04.732 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:34:14.768 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:34:24.784 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:34:34.803 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:34:44.822 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:34:54.841 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:35:04.863 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:35:14.881 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:35:24.901 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:35:34.919 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:35:44.939 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:35:54.959 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:36:04.979 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:36:14.997 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:36:25.027 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:36:35.044 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:36:45.066 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:36:55.088 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:37:05.108 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:37:15.128 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:37:25.150 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:37:35.171 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:37:45.188 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:37:55.211 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:38:05.227 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:38:15.248 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:38:25.265 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:38:35.285 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:38:45.305 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 1)... 2020-06-12T02:38:55.327 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:39:05.346 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)... 2020-06-12T02:39:15.370 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 4, have 0)...