2020-08-31T19:03:40.511 INFO:root:teuthology version: 1.0.0-1d15e8d 2020-08-31T19:03:40.513 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name pdonnell-2020-08-31_18:14:29-kcephfs-wip-pdonnell-testing-20200829.172552-distro-basic-smithi --description kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/damage} --block --lock --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2020-08-31_18:14:29-kcephfs-wip-pdonnell-testing-20200829.172552-distro-basic-smithi/5396010 -- /tmp/teuthology-worker.wwn84q12.tmp 2020-08-31T19:03:40.552 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-08-31T19:03:40.614 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2020-08-31_18:14:29-kcephfs-wip-pdonnell-testing-20200829.172552-distro-basic-smithi/5396010 branch: wip-pdonnell-testing-20200829.172552 description: kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/damage} email: pdonnell@redhat.com first_in_suite: false job_id: '5396010' kernel: client: branch: testing kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2020-08-31_18:14:29-kcephfs-wip-pdonnell-testing-20200829.172552-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: centos os_version: '8.1' overrides: admin_socket: branch: wip-pdonnell-testing-20200829.172552 ceph: cephfs_ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - Error loading MDS rank - missing journal object - Error recovering journal - error decoding table object - failed to read JournalPointer - Corrupt directory entry - Corrupt fnode header - corrupt sessionmap header - Corrupt dentry - Scrub error on inode - Metadata damage detected - MDS_READ_ONLY - force file system read-only log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: c5af0d54b89ffb9f4c441f7a4e930a89dee8fecf ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: c5af0d54b89ffb9f4c441f7a4e930a89dee8fecf selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20200829.172552 sha1: c5af0d54b89ffb9f4c441f7a4e930a89dee8fecf owner: scheduled_pdonnell@teuthology priority: 75 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.b - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - mon.c - mgr.x - mds.c - osd.4 - osd.5 - osd.6 - osd.7 - - client.0 - - client.1 - - client.2 - - client.3 sha1: c5af0d54b89ffb9f4c441f7a4e930a89dee8fecf sleep_before_teardown: 0 suite: kcephfs suite_branch: wip-pdonnell-testing-20200829.172552 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200829.172552/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: c5af0d54b89ffb9f4c441f7a4e930a89dee8fecf tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_damage teuthology_branch: rpm-test timestamp: 2020-08-31_18:14:29 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.12104 2020-08-31T19:03:40.615 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200829.172552/qa; will attempt to use it 2020-08-31T19:03:40.616 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200829.172552/qa/tasks 2020-08-31T19:03:40.616 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-08-31T19:03:40.618 INFO:teuthology.task.internal:Checking packages... 2020-08-31T19:03:40.643 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash 'c5af0d54b89ffb9f4c441f7a4e930a89dee8fecf' 2020-08-31T19:03:40.644 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-08-31T19:03:40.644 INFO:teuthology.packaging:ref: None 2020-08-31T19:03:40.645 INFO:teuthology.packaging:tag: None 2020-08-31T19:03:40.645 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20200829.172552 2020-08-31T19:03:40.645 INFO:teuthology.packaging:sha1: c5af0d54b89ffb9f4c441f7a4e930a89dee8fecf 2020-08-31T19:03:40.646 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20200829.172552 2020-08-31T19:03:40.814 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-4885.gc5af0d5 2020-08-31T19:03:40.815 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-08-31T19:03:40.822 INFO:teuthology.task.internal:no buildpackages task found 2020-08-31T19:03:40.823 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-08-31T19:03:40.837 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-08-31T19:03:40.838 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-08-31T19:03:40.890 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:03:50.909 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:04:00.938 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:04:10.985 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:04:21.010 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:04:31.035 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:04:41.055 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:04:51.075 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:05:01.099 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:05:11.120 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:05:21.141 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:05:31.161 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:05:41.181 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:05:51.203 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:06:01.222 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:06:11.242 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:06:21.262 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:06:31.280 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:06:41.304 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:06:51.326 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:07:01.345 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:07:11.364 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:07:21.387 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:07:31.410 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:07:41.438 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:07:51.461 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:08:01.484 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:08:11.527 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:08:21.548 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:08:31.567 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:08:41.586 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:08:51.606 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:09:01.635 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:09:11.652 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:09:21.673 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:09:31.700 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:09:41.731 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:09:51.750 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:10:01.771 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:10:11.797 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:10:21.819 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:10:31.846 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:10:41.865 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:10:51.884 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:11:01.924 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:11:11.954 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:11:21.980 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:11:32.001 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:11:42.021 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:11:52.040 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:12:02.063 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:12:12.091 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 4)... 2020-08-31T19:12:22.114 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:12:32.155 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:12:42.171 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:12:52.191 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:13:02.206 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:13:12.226 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:13:22.245 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:13:32.265 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:13:42.310 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:13:52.739 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:14:02.755 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:14:12.783 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:14:22.807 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:14:32.840 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:14:42.860 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:14:52.876 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:15:02.891 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:15:12.915 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:15:22.934 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:15:32.954 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:15:42.984 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:15:53.041 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:16:03.068 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:16:13.095 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:16:23.123 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:16:33.143 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:16:43.168 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:16:53.191 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:17:03.209 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:17:13.228 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:17:23.248 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:17:33.287 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:17:43.307 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:17:53.323 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:18:03.339 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:18:13.359 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:18:23.381 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:18:33.403 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:18:43.425 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:18:53.440 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:19:03.452 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:19:13.469 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:19:23.497 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:19:33.518 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:19:43.541 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:19:53.559 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:20:03.585 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 3)... 2020-08-31T19:20:13.607 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:20:23.627 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:20:33.647 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 4)... 2020-08-31T19:20:43.665 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 3)... 2020-08-31T19:20:53.686 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:21:03.708 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:21:13.728 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:21:23.777 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:21:33.800 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:21:43.820 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:21:53.835 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:22:03.860 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:22:13.882 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:22:23.911 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:22:33.937 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:22:43.967 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:22:54.008 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 3)... 2020-08-31T19:23:04.029 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:23:14.048 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:23:24.069 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:23:34.090 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:23:44.109 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:23:54.130 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:24:04.152 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:24:14.174 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:24:24.191 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:24:34.209 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:24:44.232 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:24:54.253 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:25:04.275 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:25:14.295 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:25:24.315 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:25:34.333 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:25:44.354 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:25:54.379 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:26:04.611 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:26:14.633 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:26:24.654 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:26:34.672 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:26:44.690 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:26:54.709 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:27:04.728 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:27:14.749 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:27:24.776 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:27:34.806 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:27:44.830 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:27:54.850 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:28:04.867 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:28:14.892 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:28:24.914 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:28:34.935 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 4)... 2020-08-31T19:28:44.955 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:28:54.973 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:29:04.999 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:29:15.021 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:29:25.042 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 5)... 2020-08-31T19:29:35.063 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:29:45.079 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:29:55.100 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:30:05.119 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:30:15.139 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:30:25.162 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:30:35.189 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:30:45.211 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:30:55.234 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:31:05.255 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:31:15.290 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 4)... 2020-08-31T19:31:25.314 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:31:35.340 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:31:45.362 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:31:55.389 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:32:05.413 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:32:15.435 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:32:25.454 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:32:35.470 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:32:45.496 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:32:55.518 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:33:05.541 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:33:15.562 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:33:25.581 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:33:35.603 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:33:45.621 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:33:55.642 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:34:05.662 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:34:15.682 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:34:25.707 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:34:35.732 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:34:45.760 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:34:55.786 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:35:05.804 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 4)... 2020-08-31T19:35:15.834 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:35:25.853 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:35:35.874 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:35:45.892 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:35:55.913 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:36:05.932 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:36:15.953 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:36:25.973 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:36:35.996 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:36:46.017 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:36:56.036 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 3)... 2020-08-31T19:37:06.057 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:37:16.081 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 3)... 2020-08-31T19:37:26.105 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 3)... 2020-08-31T19:37:36.128 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 3)... 2020-08-31T19:37:46.155 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 3)... 2020-08-31T19:37:56.183 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:38:06.202 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:38:16.221 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:38:26.244 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:38:36.272 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:38:46.295 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:38:56.313 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:39:06.336 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 2)... 2020-08-31T19:39:16.358 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:39:26.379 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:39:36.399 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:39:46.421 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:39:56.440 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:40:06.462 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:40:16.484 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:40:26.514 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:40:36.534 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:40:46.558 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:40:56.573 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:41:06.602 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:41:16.621 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:41:26.641 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:41:36.662 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:41:46.693 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:41:56.730 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:42:06.749 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:42:16.771 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:42:26.792 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 3)... 2020-08-31T19:42:36.813 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:42:46.835 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:42:56.855 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:43:06.883 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:43:16.910 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:43:26.928 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:43:36.951 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:43:46.969 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:43:56.991 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:44:07.011 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:44:17.035 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 0)... 2020-08-31T19:44:27.051 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:44:37.069 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:44:47.091 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:44:57.111 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:45:07.131 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:45:17.164 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:45:27.193 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:45:37.215 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:45:47.251 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:45:57.274 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:46:07.289 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:46:17.311 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)... 2020-08-31T19:46:27.328 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 6, have 1)...