2021-01-25T02:37:39.989 INFO:root:teuthology version: 1.0.0-fe79ba1b 2021-01-25T02:37:39.991 DEBUG:teuthology.run:Teuthology command: teuthology --block --lock --description upgrade:luminous-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/luminous 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 6-msgr2 6-nautilus 7-final-workload/{rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/centos_latest thrashosds-health} --verbose --name teuthology-2021-01-25_02:25:03-upgrade:luminous-x-nautilus-distro-basic-smithi --archive /home/teuthworker/archive/teuthology-2021-01-25_02:25:03-upgrade:luminous-x-nautilus-distro-basic-smithi/5823645 --owner scheduled_teuthology@teuthology -- /tmp/teuthology-worker.lqc10krm.tmp 2021-01-25T02:37:40.076 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-01-25T02:37:40.163 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2021-01-25_02:25:03-upgrade:luminous-x-nautilus-distro-basic-smithi/5823645 branch: nautilus description: upgrade:luminous-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/luminous 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 5-finish-upgrade 6-msgr2 6-nautilus 7-final-workload/{rgw-swift snaps-many-objects} objectstore/bluestore-bitmap supported-all-distro/centos_latest thrashosds-health} email: ceph-qa@ceph.io first_in_suite: false job_id: '5823645' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'Run ceph on two nodes, with a separate client-only node. Use xfs beneath the osds. ' - desc: install ceph/luminous latest - desc: 'install upgrade ceph/-x on 2/3 of cluster restart : mons, osd.0-7 ' - desc: 'randomly kill and revive osd small chance to increase the number of pgs ' - desc: 'run randomized correctness test for rados operations generate write load with rados bench ' - desc: 'run basic cls tests for rbd ' - desc: 'run basic import/export cli tests for rbd ' - desc: 'librbd C and C++ api tests ' - desc: 'randomized correctness test for rados operations on a replicated pool, using only reads, writes, and deletes ' - desc: 'randomized correctness test for rados operations on a replicated pool with snapshot operations ' - desc: 'swift api tests for rgw ' - desc: 'randomized correctness test for rados operations on a replicated pool with snapshot operations ' name: teuthology-2021-01-25_02:25:03-upgrade:luminous-x-nautilus-distro-basic-smithi nuke-on-error: true openstack: - machine: disk: 100 - volumes: count: 4 size: 30 os_type: centos os_version: '7.8' overrides: admin_socket: branch: nautilus ceph: conf: global: enable experimental unrecoverable data corrupting features: '*' mon warn on msgr2 not enabled: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on osd down out interval zero: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd max pg log entries: 2 osd min pg log entries: 1 osd objectstore: bluestore fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(MON_DOWN\) - \(MGR_DOWN\) - slow request - \(MON_MSGR2_NOT_ENABLED\) - but it is still running - wrongly marked me down - objects unfound and apparently lost - log bound mismatch - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request mon_bind_addrvec: false mon_bind_msgr2: false sha1: a39a5e4772ef4262056deb8e2ee387ee2719bfc0 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: sha1: a39a5e4772ef4262056deb8e2ee387ee2719bfc0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: nautilus sha1: a39a5e4772ef4262056deb8e2ee387ee2719bfc0 owner: scheduled_teuthology@teuthology priority: 50 repo: git://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - osd.4 - osd.5 - osd.6 - osd.7 - - mon.c - - osd.8 - osd.9 - osd.10 - osd.11 - - client.0 sha1: a39a5e4772ef4262056deb8e2ee387ee2719bfc0 sleep_before_teardown: 0 stress-tasks: - thrashosds: aggressive_pg_num_changes: false chance_force_recovery: 0 chance_pgnum_grow: 1 chance_pgpnum_fix: 1 chance_thrash_cluster_full: 0 chance_thrash_pg_upmap: 0 chance_thrash_pg_upmap_items: 0 disable_objectstore_tool_tests: true timeout: 1200 - print: '**** done thrashosds 3-thrash' - full_sequential: - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - print: '**** done radosbench 7-workload' - workunit: branch: luminous clients: client.0: - cls/test_cls_rbd.sh - print: '**** done cls/test_cls_rbd.sh 5-workload' - workunit: branch: luminous clients: client.0: - rbd/import_export.sh env: RBD_CREATE_ARGS: --new-format - print: '**** done rbd/import_export.sh 5-workload' - workunit: branch: luminous clients: client.0: - rbd/test_librbd.sh - print: '**** done rbd/test_librbd.sh 7-workload' - full_sequential: - rados: clients: - client.0 objects: 500 op_weights: delete: 10 read: 45 write: 45 ops: 4000 write_append_excl: false - print: '**** done rados/readwrite 5-workload' - full_sequential: - rados: clients: - client.0 objects: 50 op_weights: delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 write_append_excl: false - print: '**** done rados/snaps-few-objects 5-workload' suite: upgrade:luminous-x suite_branch: nautilus suite_path: /home/teuthworker/src/git.ceph.com_ceph_nautilus/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph.git suite_sha1: a39a5e4772ef4262056deb8e2ee387ee2719bfc0 tasks: - install: branch: luminous exclude_packages: - librados3 - ceph-mgr-dashboard - ceph-mgr-diskprediction-local - ceph-mgr-diskprediction-cloud - ceph-mgr-rook - ceph-mgr-ssh extra_packages: - librados2 - print: '**** done install luminous' - ceph: conf: global: bluestore_warn_on_legacy_statfs: false mon pg warn min per osd: 0 - exec: osd.0: - ceph osd require-osd-release luminous - ceph osd set-require-min-compat-client luminous - print: '**** done ceph' - install.upgrade: mon.a: null mon.b: null mon.c: null - print: '**** done install.upgrade of first 3 nodes' - ceph.restart: daemons: - mon.a - mon.b - mgr.x - osd.0 - osd.1 - osd.2 - osd.3 - osd.4 - osd.5 - osd.6 - osd.7 mon-health-to-clog: false - print: '**** done ceph.restart of all mons and 2/3 of osds' - parallel: - stress-tasks - install.upgrade: client.0: null osd.8: null - ceph.restart: daemons: - mon.c - osd.8 - osd.9 - osd.10 - osd.11 wait-for-healthy: false wait-for-osds-up: true - exec: osd.0: - ceph osd set pglog_hardlimit - ceph osd dump --format=json-pretty | grep "flags" - ceph config set global mon_warn_on_msgr2_not_enabled false - print: '**** try to set pglog_hardlimit again, should succeed' - exec: mon.a: - ceph mon enable-msgr2 - ceph config rm global mon_warn_on_msgr2_not_enabled - ceph.healthy: null - exec: osd.0: - ceph osd require-osd-release nautilus - ceph osd set-require-min-compat-client nautilus - ceph.healthy: null - rgw: client.0: null - print: '**** done rgw 9-workload' - swift: client.0: force-branch: ceph-nautilus rgw_server: client.0 - print: '**** done swift 9-workload' - rados: clients: - client.0 objects: 500 op_weights: delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 write_append_excl: false teuthology_branch: master timestamp: 2021-01-25_02:25:03 tube: smithi user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.6147 2021-01-25T02:37:40.164 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_nautilus/qa; will attempt to use it 2021-01-25T02:37:40.164 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_nautilus/qa/tasks 2021-01-25T02:37:40.165 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-01-25T02:37:40.167 INFO:teuthology.task.internal:Checking packages... 2021-01-25T02:37:40.199 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash 'a39a5e4772ef4262056deb8e2ee387ee2719bfc0' 2021-01-25T02:37:40.199 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-01-25T02:37:40.200 INFO:teuthology.packaging:ref: None 2021-01-25T02:37:40.201 INFO:teuthology.packaging:tag: None 2021-01-25T02:37:40.201 INFO:teuthology.packaging:branch: nautilus 2021-01-25T02:37:40.202 INFO:teuthology.packaging:sha1: a39a5e4772ef4262056deb8e2ee387ee2719bfc0 2021-01-25T02:37:40.203 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&ref=nautilus 2021-01-25T02:37:40.506 INFO:teuthology.task.internal:Found packages for ceph version 14.2.16-204.ga39a5e47 2021-01-25T02:37:40.506 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-01-25T02:37:40.513 INFO:teuthology.task.internal:no buildpackages task found 2021-01-25T02:37:40.514 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2021-01-25T02:37:40.589 INFO:teuthology.task.internal.lock_machines:Locking machines... 2021-01-25T02:37:40.589 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-01-25T02:37:40.639 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:37:50.653 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:38:00.670 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:38:10.685 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:38:20.703 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:38:30.729 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 4)... 2021-01-25T02:38:40.747 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:38:50.772 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:39:00.790 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:39:10.806 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:39:20.823 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:39:30.840 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:39:40.857 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:39:50.874 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:40:00.890 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:40:10.907 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:40:20.927 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:40:30.943 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:40:40.960 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:40:50.976 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:41:00.989 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:41:11.006 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:41:21.025 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:41:31.045 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:41:41.062 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:41:51.080 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:42:01.095 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:42:11.110 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:42:21.127 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:42:31.144 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:42:41.165 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:42:51.185 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:43:01.207 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:43:11.318 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:43:21.333 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:43:31.350 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:43:41.366 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:43:51.381 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:44:01.409 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:44:11.548 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:44:21.562 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:44:31.578 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:44:41.594 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:44:51.610 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:45:01.630 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:45:11.647 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:45:21.663 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:45:31.680 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:45:41.698 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:45:51.715 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:46:01.723 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:46:11.737 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:46:21.755 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T02:46:31.771 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:46:41.787 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:46:51.804 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:47:01.820 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:47:11.833 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:47:21.850 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:47:31.865 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:47:41.881 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:47:51.897 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:48:01.913 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:48:11.933 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:48:21.952 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:48:31.972 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T02:48:41.988 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:48:52.001 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:49:02.017 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:49:12.033 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:49:22.043 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:49:32.060 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:49:42.078 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:49:52.096 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:50:02.123 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:50:12.139 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:50:22.154 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:50:32.171 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:50:42.188 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:50:52.211 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:51:02.502 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:51:12.519 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:51:22.534 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:51:32.550 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:51:42.565 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:51:52.581 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:52:02.597 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:52:12.614 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:52:22.628 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:52:32.646 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:52:42.664 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:52:52.678 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:53:02.694 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:53:12.712 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:53:22.726 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:53:32.741 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:53:42.759 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:53:52.774 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:54:02.792 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:54:12.810 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:54:22.827 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:54:32.841 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:54:42.858 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:54:52.874 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:55:02.891 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:55:12.907 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:55:22.921 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:55:32.937 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:55:42.953 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:55:52.971 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:56:02.986 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:56:13.002 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:56:23.017 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:56:33.034 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:56:43.050 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:56:53.068 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:57:03.082 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:57:13.098 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:57:23.113 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:57:33.130 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:57:43.146 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:57:53.162 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:58:03.177 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:58:13.193 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:58:23.210 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:58:33.226 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:58:43.241 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:58:53.258 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:59:03.273 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:59:13.289 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:59:23.304 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:59:33.318 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:59:43.336 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T02:59:53.350 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T03:00:03.366 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:00:13.382 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:00:23.399 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:00:33.414 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:00:43.430 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:00:53.448 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:01:03.465 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:01:13.482 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 2)... 2021-01-25T03:01:23.499 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:01:33.520 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:01:43.563 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:01:53.580 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:02:03.598 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:02:13.614 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:02:23.636 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:02:33.651 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:02:43.667 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:02:53.683 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:03:03.699 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:03:13.715 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:03:23.729 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:03:33.746 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:03:43.764 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:03:53.783 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:04:03.799 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:04:13.815 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:04:23.829 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:04:33.847 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:04:43.863 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:04:53.878 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:05:03.895 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:05:13.910 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:05:23.926 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:05:33.942 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:05:43.955 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:05:53.970 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:06:03.988 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:06:14.004 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:06:24.019 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:06:34.035 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:06:44.050 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:06:54.069 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:07:04.087 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:07:14.107 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:07:24.124 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:07:34.142 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:07:44.160 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:07:54.181 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:08:04.198 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:08:14.216 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:08:24.227 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:08:34.237 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:08:44.255 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:08:54.269 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:09:04.287 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:09:14.301 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:09:24.318 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:09:34.334 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:09:44.349 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:09:54.367 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:10:04.383 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:10:14.397 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:10:24.415 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:10:34.431 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:10:44.447 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:10:54.455 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:11:04.470 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:11:14.483 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:11:24.499 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:11:34.514 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:11:44.529 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:11:54.547 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:12:04.606 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:12:14.622 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:12:24.635 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:12:34.650 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:12:44.662 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:12:54.680 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:13:04.690 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:13:14.701 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:13:24.718 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:13:34.734 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:13:44.752 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:13:54.769 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:14:04.791 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:14:14.807 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:14:24.822 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:14:34.839 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:14:44.854 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:14:54.870 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:15:04.887 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:15:14.901 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:15:24.918 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:15:34.937 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:15:44.954 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:15:54.970 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:16:04.988 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:16:15.004 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:16:25.019 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:16:35.036 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:16:45.051 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:16:55.069 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:17:05.086 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:17:15.102 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:17:25.119 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:17:35.142 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:17:45.157 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:17:55.174 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:18:05.191 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:18:15.208 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:18:25.222 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:18:35.240 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:18:45.254 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:18:55.269 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:19:05.286 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:19:15.304 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:19:25.318 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:19:35.335 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:19:45.354 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:19:55.370 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:20:05.385 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:20:15.415 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:20:25.431 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:20:35.447 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:20:45.462 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:20:55.479 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:21:05.494 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:21:15.511 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 4)... 2021-01-25T03:21:25.527 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:21:35.541 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:21:45.552 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:21:55.569 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:22:05.586 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:22:15.602 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:22:25.618 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:22:35.633 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:22:45.654 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:22:55.668 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:23:05.684 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:23:15.701 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:23:25.715 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:23:35.725 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:23:45.742 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:23:55.759 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:24:05.774 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:24:15.790 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:24:25.806 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:24:35.822 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:24:45.840 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:24:55.853 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:25:05.870 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:25:15.889 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:25:25.907 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:25:35.932 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:25:45.949 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:25:55.968 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:26:05.982 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:26:15.998 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:26:26.052 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:26:36.068 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:26:46.077 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:26:56.094 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:27:06.118 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:27:16.136 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:27:26.153 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:27:36.171 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:27:46.189 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:27:56.207 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:28:06.222 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:28:16.241 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:28:26.258 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:28:36.275 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:28:46.290 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:28:56.308 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:29:06.322 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:29:16.338 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:29:26.354 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:29:36.370 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:29:46.387 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:29:56.403 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:30:06.420 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:30:16.440 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:30:26.459 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:30:36.473 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:30:46.489 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:30:56.507 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:31:06.524 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:31:16.540 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:31:26.559 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:31:36.587 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:31:46.607 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:31:56.627 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:32:06.643 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:32:16.669 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:32:26.687 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:32:36.719 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:32:46.734 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:32:56.749 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:33:06.766 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:33:16.781 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:33:26.805 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:33:36.821 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:33:46.838 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:33:56.852 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:34:06.869 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:34:16.886 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:34:26.901 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:34:36.917 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:34:46.935 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:34:56.950 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:35:06.975 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:35:16.989 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:35:27.006 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:35:37.059 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:35:47.074 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:35:57.094 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:36:07.114 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:36:17.137 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 1)... 2021-01-25T03:36:27.156 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:36:37.171 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:36:47.192 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:36:57.209 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:37:07.226 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:37:17.235 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:37:27.254 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:37:37.271 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:37:47.287 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:37:57.303 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:38:07.319 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:38:17.334 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:38:27.352 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:38:37.367 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:38:47.392 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:38:57.409 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:39:07.426 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:39:17.444 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:39:27.460 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:39:37.473 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:39:47.490 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:39:57.506 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:40:07.525 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:40:17.536 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:40:27.550 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:40:37.565 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:40:47.583 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:40:57.601 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:41:07.619 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:41:17.636 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:41:27.652 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:41:37.664 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:41:47.684 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:41:57.702 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:42:07.719 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 5, have 0)... 2021-01-25T03:42:17.735 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_teuthology@teuthology', 'count': 5, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/teuthology-2021-01-25_02:25:03-upgrade:luminous-x-nautilus-distro-basic-smithi/5823645'} 2021-01-25T03:42:17.761 DEBUG:teuthology.lock.ops:locked ubuntu@smithi008.front.sepia.ceph.com, ubuntu@smithi197.front.sepia.ceph.com, ubuntu@smithi125.front.sepia.ceph.com, ubuntu@smithi171.front.sepia.ceph.com, ubuntu@smithi192.front.sepia.ceph.com 2021-01-25T03:42:17.885 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f59671f0840>, signals=[15]) 2021-01-25T03:42:18.107 INFO:teuthology.provision.fog.smithi197:Scheduling deploy of centos 7.8 2021-01-25T03:42:18.119 INFO:teuthology.provision.fog.smithi125:Scheduling deploy of centos 7.8 2021-01-25T03:42:18.154 INFO:teuthology.provision.fog.smithi008:Scheduling deploy of centos 7.8 2021-01-25T03:42:18.180 INFO:teuthology.provision.fog.smithi171:Scheduling deploy of centos 7.8 2021-01-25T03:42:18.204 INFO:teuthology.provision.fog.smithi192:Scheduling deploy of centos 7.8 2021-01-25T03:42:18.493 INFO:teuthology.orchestra.console:Power off smithi125 2021-01-25T03:42:18.494 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2021-01-25T03:42:18.519 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:18.533 INFO:teuthology.orchestra.console:Power off smithi008 2021-01-25T03:42:18.534 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2021-01-25T03:42:18.547 INFO:teuthology.orchestra.console:Power off smithi197 2021-01-25T03:42:18.547 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi197.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2021-01-25T03:42:18.560 INFO:teuthology.orchestra.console:Power off smithi171 2021-01-25T03:42:18.561 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2021-01-25T03:42:18.568 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi197.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:18.576 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:18.590 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:18.608 INFO:teuthology.orchestra.console:Power off smithi192 2021-01-25T03:42:18.609 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2021-01-25T03:42:18.629 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:18.668 INFO:teuthology.orchestra.console:Power off for smithi125 completed 2021-01-25T03:42:18.700 INFO:teuthology.orchestra.console:Power off for smithi197 completed 2021-01-25T03:42:18.702 INFO:teuthology.orchestra.console:Power off for smithi008 completed 2021-01-25T03:42:18.718 INFO:teuthology.orchestra.console:Power off for smithi171 completed 2021-01-25T03:42:18.750 INFO:teuthology.orchestra.console:Power off for smithi192 completed 2021-01-25T03:42:18.769 INFO:teuthology.orchestra.console:Power on smithi125 2021-01-25T03:42:18.770 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2021-01-25T03:42:18.790 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:18.801 INFO:teuthology.orchestra.console:Power on smithi197 2021-01-25T03:42:18.802 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi197.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2021-01-25T03:42:18.808 INFO:teuthology.orchestra.console:Power on smithi008 2021-01-25T03:42:18.809 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2021-01-25T03:42:18.819 INFO:teuthology.orchestra.console:Power on smithi171 2021-01-25T03:42:18.820 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2021-01-25T03:42:18.829 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:18.836 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi197.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:18.843 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:18.850 INFO:teuthology.orchestra.console:Power on smithi192 2021-01-25T03:42:18.851 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2021-01-25T03:42:18.869 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:22.819 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:22.857 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:22.865 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi197.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:22.872 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:22.904 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:23.030 INFO:teuthology.orchestra.console:Power on for smithi125 completed 2021-01-25T03:42:23.067 INFO:teuthology.orchestra.console:Power on for smithi008 completed 2021-01-25T03:42:23.074 INFO:teuthology.orchestra.console:Power on for smithi197 completed 2021-01-25T03:42:23.131 INFO:teuthology.provision.fog.smithi125:Waiting for deploy to finish 2021-01-25T03:42:23.168 INFO:teuthology.provision.fog.smithi008:Waiting for deploy to finish 2021-01-25T03:42:23.174 INFO:teuthology.provision.fog.smithi197:Waiting for deploy to finish 2021-01-25T03:42:26.983 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:27.014 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2021-01-25T03:42:27.194 INFO:teuthology.orchestra.console:Power on for smithi171 completed 2021-01-25T03:42:27.223 INFO:teuthology.orchestra.console:Power on for smithi192 completed 2021-01-25T03:42:27.296 INFO:teuthology.provision.fog.smithi171:Waiting for deploy to finish 2021-01-25T03:42:27.324 INFO:teuthology.provision.fog.smithi192:Waiting for deploy to finish 2021-01-25T03:44:09.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:44:18.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:44:27.413 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:44:36.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:44:45.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:44:54.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:44:54.566 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:44:54.612 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:44:58.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:03.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:03.639 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:03.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:07.573 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:12.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:12.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:12.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:13.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:16.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:21.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:21.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:21.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:22.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:25.684 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:30.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:30.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:30.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:31.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:34.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:39.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:39.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:40.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:43.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:47.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:48.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:48.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:50.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:52.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:55.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:55.180 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:45:56.278 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:45:57.377 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:45:57.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:58.474 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:45:59.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:45:59.622 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:00.726 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:01.784 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:02.884 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:03.981 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:05.040 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:08.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:46:11.042 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:46:11.100 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:12.103 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-01-25T03:46:12.104 ERROR:paramiko.transport:Traceback (most recent call last): 2021-01-25T03:46:12.105 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-01-25T03:46:12.105 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-01-25T03:46:12.105 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-01-25T03:46:12.106 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-01-25T03:46:12.106 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-01-25T03:46:12.106 ERROR:paramiko.transport: raise EOFError() 2021-01-25T03:46:12.107 ERROR:paramiko.transport:EOFError 2021-01-25T03:46:12.107 ERROR:paramiko.transport: 2021-01-25T03:46:12.107 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-01-25T03:46:12.107 ERROR:paramiko.transport: 2021-01-25T03:46:12.108 ERROR:paramiko.transport:Traceback (most recent call last): 2021-01-25T03:46:12.108 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-01-25T03:46:12.108 ERROR:paramiko.transport: self._check_banner() 2021-01-25T03:46:12.109 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-01-25T03:46:12.109 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-01-25T03:46:12.109 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-01-25T03:46:12.109 ERROR:paramiko.transport: 2021-01-25T03:46:17.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:46:18.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:46:18.168 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:19.171 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-01-25T03:46:19.172 ERROR:paramiko.transport:Traceback (most recent call last): 2021-01-25T03:46:19.172 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-01-25T03:46:19.172 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-01-25T03:46:19.173 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-01-25T03:46:19.173 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-01-25T03:46:19.173 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-01-25T03:46:19.174 ERROR:paramiko.transport: raise EOFError() 2021-01-25T03:46:19.174 ERROR:paramiko.transport:EOFError 2021-01-25T03:46:19.174 ERROR:paramiko.transport: 2021-01-25T03:46:19.175 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-01-25T03:46:19.175 ERROR:paramiko.transport: 2021-01-25T03:46:19.175 ERROR:paramiko.transport:Traceback (most recent call last): 2021-01-25T03:46:19.176 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-01-25T03:46:19.176 ERROR:paramiko.transport: self._check_banner() 2021-01-25T03:46:19.176 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-01-25T03:46:19.177 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-01-25T03:46:19.177 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-01-25T03:46:19.177 ERROR:paramiko.transport: 2021-01-25T03:46:25.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:46:25.328 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:26.386 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:27.484 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:28.584 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:29.587 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-01-25T03:46:29.587 ERROR:paramiko.transport:Traceback (most recent call last): 2021-01-25T03:46:29.588 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-01-25T03:46:29.588 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-01-25T03:46:29.588 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-01-25T03:46:29.589 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-01-25T03:46:29.589 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-01-25T03:46:29.589 ERROR:paramiko.transport: raise EOFError() 2021-01-25T03:46:29.589 ERROR:paramiko.transport:EOFError 2021-01-25T03:46:29.590 ERROR:paramiko.transport: 2021-01-25T03:46:29.590 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-01-25T03:46:29.590 ERROR:paramiko.transport: 2021-01-25T03:46:29.591 ERROR:paramiko.transport:Traceback (most recent call last): 2021-01-25T03:46:29.591 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-01-25T03:46:29.591 ERROR:paramiko.transport: self._check_banner() 2021-01-25T03:46:29.591 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-01-25T03:46:29.592 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-01-25T03:46:29.592 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-01-25T03:46:29.592 ERROR:paramiko.transport: 2021-01-25T03:46:30.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:46:30.539 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-01-25T03:46:30.540 DEBUG:teuthology.orchestra.run.smithi192:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2021-01-25T03:46:35.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:46:35.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:46:35.635 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-01-25T03:46:35.635 DEBUG:teuthology.orchestra.run.smithi125:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2021-01-25T03:46:35.651 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:36.750 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:37.847 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:38.850 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-01-25T03:46:38.850 ERROR:paramiko.transport:Traceback (most recent call last): 2021-01-25T03:46:38.850 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-01-25T03:46:38.851 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-01-25T03:46:38.851 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-01-25T03:46:38.851 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-01-25T03:46:38.852 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-01-25T03:46:38.852 ERROR:paramiko.transport: raise EOFError() 2021-01-25T03:46:38.852 ERROR:paramiko.transport:EOFError 2021-01-25T03:46:38.852 ERROR:paramiko.transport: 2021-01-25T03:46:38.853 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-01-25T03:46:38.853 ERROR:paramiko.transport: 2021-01-25T03:46:38.853 ERROR:paramiko.transport:Traceback (most recent call last): 2021-01-25T03:46:38.854 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-01-25T03:46:38.854 ERROR:paramiko.transport: self._check_banner() 2021-01-25T03:46:38.854 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-01-25T03:46:38.855 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-01-25T03:46:38.855 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-01-25T03:46:38.855 ERROR:paramiko.transport: 2021-01-25T03:46:41.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:46:44.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:46:44.858 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-01-25T03:46:44.859 ERROR:paramiko.transport:Traceback (most recent call last): 2021-01-25T03:46:44.859 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-01-25T03:46:44.859 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-01-25T03:46:44.860 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-01-25T03:46:44.860 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-01-25T03:46:44.860 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-01-25T03:46:44.860 ERROR:paramiko.transport: raise EOFError() 2021-01-25T03:46:44.861 ERROR:paramiko.transport:EOFError 2021-01-25T03:46:44.861 ERROR:paramiko.transport: 2021-01-25T03:46:44.861 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-01-25T03:46:44.862 ERROR:paramiko.transport: 2021-01-25T03:46:44.862 ERROR:paramiko.transport:Traceback (most recent call last): 2021-01-25T03:46:44.862 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-01-25T03:46:44.863 ERROR:paramiko.transport: self._check_banner() 2021-01-25T03:46:44.863 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-01-25T03:46:44.863 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-01-25T03:46:44.864 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-01-25T03:46:44.864 ERROR:paramiko.transport: 2021-01-25T03:46:50.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:46:50.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:46:50.964 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:51.083 INFO:teuthology.provision.fog.smithi192:Node is ready 2021-01-25T03:46:51.084 DEBUG:teuthology.orchestra.run.smithi192:> hostname 2021-01-25T03:46:51.207 INFO:teuthology.orchestra.run.smithi192.stdout:smithi192.front.sepia.ceph.com 2021-01-25T03:46:51.208 DEBUG:teuthology.orchestra.run.smithi192:> grep smithi192.front.sepia.ceph.com /etc/hosts 2021-01-25T03:46:51.289 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.15.192 smithi192.front.sepia.ceph.com smithi192 2021-01-25T03:46:51.290 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname smithi192 2021-01-25T03:46:51.455 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i -e 's/smithi192.front.sepia.ceph.com/smithi192/g' /etc/hosts 2021-01-25T03:46:51.545 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i -e 's/172.21.15.192/172.21.15.192/g' /etc/hosts 2021-01-25T03:46:51.625 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i -e 's/smithi192.front.sepia.ceph.com/smithi192/g' /etc/hostname 2021-01-25T03:46:51.691 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname smithi192 2021-01-25T03:46:51.753 INFO:teuthology.provision.fog.smithi192:Deploy complete! 2021-01-25T03:46:51.754 DEBUG:teuthology.parallel:result is None 2021-01-25T03:46:52.061 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:53.158 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:54.259 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:54.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:46:55.167 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-01-25T03:46:55.168 DEBUG:teuthology.orchestra.run.smithi008:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2021-01-25T03:46:55.355 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:55.771 INFO:teuthology.provision.fog.smithi008:Node is ready 2021-01-25T03:46:55.772 DEBUG:teuthology.orchestra.run.smithi008:> hostname 2021-01-25T03:46:55.857 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008.front.sepia.ceph.com 2021-01-25T03:46:55.858 DEBUG:teuthology.orchestra.run.smithi008:> grep smithi008.front.sepia.ceph.com /etc/hosts 2021-01-25T03:46:55.935 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8 smithi008.front.sepia.ceph.com smithi008 2021-01-25T03:46:55.936 DEBUG:teuthology.orchestra.run.smithi008:> sudo hostname smithi008 2021-01-25T03:46:56.093 INFO:teuthology.provision.fog.smithi125:Node is ready 2021-01-25T03:46:56.175 DEBUG:teuthology.orchestra.run.smithi125:> hostname 2021-01-25T03:46:56.179 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i -e 's/smithi008.front.sepia.ceph.com/smithi008/g' /etc/hosts 2021-01-25T03:46:56.301 INFO:teuthology.orchestra.run.smithi125.stdout:smithi125.front.sepia.ceph.com 2021-01-25T03:46:56.302 DEBUG:teuthology.orchestra.run.smithi125:> grep smithi125.front.sepia.ceph.com /etc/hosts 2021-01-25T03:46:56.316 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i -e 's/172.21.15.8/172.21.15.8/g' /etc/hosts 2021-01-25T03:46:56.389 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.125 smithi125.front.sepia.ceph.com smithi125 2021-01-25T03:46:56.389 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname smithi125 2021-01-25T03:46:56.438 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i -e 's/smithi008.front.sepia.ceph.com/smithi008/g' /etc/hostname 2021-01-25T03:46:56.452 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:46:56.547 DEBUG:teuthology.orchestra.run.smithi008:> sudo hostname smithi008 2021-01-25T03:46:56.552 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/smithi125.front.sepia.ceph.com/smithi125/g' /etc/hosts 2021-01-25T03:46:56.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:46:56.661 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/172.21.15.125/172.21.15.125/g' /etc/hosts 2021-01-25T03:46:56.665 INFO:teuthology.provision.fog.smithi008:Deploy complete! 2021-01-25T03:46:56.666 DEBUG:teuthology.parallel:result is None 2021-01-25T03:46:56.759 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/smithi125.front.sepia.ceph.com/smithi125/g' /etc/hostname 2021-01-25T03:46:56.841 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname smithi125 2021-01-25T03:46:56.870 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-01-25T03:46:56.871 DEBUG:teuthology.orchestra.run.smithi171:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2021-01-25T03:46:56.912 INFO:teuthology.provision.fog.smithi125:Deploy complete! 2021-01-25T03:46:56.913 DEBUG:teuthology.parallel:result is None 2021-01-25T03:46:57.455 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-01-25T03:46:57.455 ERROR:paramiko.transport:Traceback (most recent call last): 2021-01-25T03:46:57.456 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-01-25T03:46:57.456 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-01-25T03:46:57.456 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-01-25T03:46:57.457 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-01-25T03:46:57.457 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-01-25T03:46:57.457 ERROR:paramiko.transport: raise EOFError() 2021-01-25T03:46:57.458 ERROR:paramiko.transport:EOFError 2021-01-25T03:46:57.458 ERROR:paramiko.transport: 2021-01-25T03:46:57.458 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-01-25T03:46:57.459 ERROR:paramiko.transport: 2021-01-25T03:46:57.459 ERROR:paramiko.transport:Traceback (most recent call last): 2021-01-25T03:46:57.459 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-01-25T03:46:57.460 ERROR:paramiko.transport: self._check_banner() 2021-01-25T03:46:57.460 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-01-25T03:46:57.460 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-01-25T03:46:57.460 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-01-25T03:46:57.461 ERROR:paramiko.transport: 2021-01-25T03:47:03.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:47:03.560 ERROR:teuthology.orchestra.connection:Error authenticating with smithi197.front.sepia.ceph.com: Authentication failed. 2021-01-25T03:47:04.563 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-01-25T03:47:04.564 ERROR:paramiko.transport:Traceback (most recent call last): 2021-01-25T03:47:04.564 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-01-25T03:47:04.564 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-01-25T03:47:04.565 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-01-25T03:47:04.565 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-01-25T03:47:04.565 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-01-25T03:47:04.566 ERROR:paramiko.transport: raise EOFError() 2021-01-25T03:47:04.566 ERROR:paramiko.transport:EOFError 2021-01-25T03:47:04.566 ERROR:paramiko.transport: 2021-01-25T03:47:04.567 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-01-25T03:47:04.567 ERROR:paramiko.transport: 2021-01-25T03:47:04.567 ERROR:paramiko.transport:Traceback (most recent call last): 2021-01-25T03:47:04.568 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-01-25T03:47:04.568 ERROR:paramiko.transport: self._check_banner() 2021-01-25T03:47:04.568 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-01-25T03:47:04.568 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-01-25T03:47:04.569 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-01-25T03:47:04.569 ERROR:paramiko.transport: 2021-01-25T03:47:10.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:47:12.350 INFO:teuthology.provision.fog.smithi171:Node is ready 2021-01-25T03:47:12.351 DEBUG:teuthology.orchestra.run.smithi171:> hostname 2021-01-25T03:47:12.485 INFO:teuthology.orchestra.run.smithi171.stdout:smithi061 2021-01-25T03:47:12.485 DEBUG:teuthology.orchestra.run.smithi171:> grep smithi061 /etc/hosts 2021-01-25T03:47:12.568 INFO:teuthology.orchestra.run.smithi171.stdout:172.21.15.61 smithi061.front.sepia.ceph.com smithi061 2021-01-25T03:47:12.569 DEBUG:teuthology.orchestra.run.smithi171:> sudo hostname smithi171 2021-01-25T03:47:12.724 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i -e 's/smithi061/smithi171/g' /etc/hosts 2021-01-25T03:47:12.807 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i -e 's/172.21.15.61/172.21.15.171/g' /etc/hosts 2021-01-25T03:47:12.885 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i -e 's/smithi061/smithi171/g' /etc/hostname 2021-01-25T03:47:12.956 DEBUG:teuthology.orchestra.run.smithi171:> sudo hostname smithi171 2021-01-25T03:47:13.022 INFO:teuthology.provision.fog.smithi171:Deploy complete! 2021-01-25T03:47:13.022 DEBUG:teuthology.parallel:result is None 2021-01-25T03:47:51.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:47:58.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:48:07.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:48:16.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:49:22.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:49:22.592 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-01-25T03:49:22.593 DEBUG:teuthology.orchestra.run.smithi197:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2021-01-25T03:49:23.026 INFO:teuthology.provision.fog.smithi197:Node is ready 2021-01-25T03:49:23.026 DEBUG:teuthology.orchestra.run.smithi197:> hostname 2021-01-25T03:49:23.073 INFO:teuthology.orchestra.run.smithi197.stdout:smithi197.front.sepia.ceph.com 2021-01-25T03:49:23.073 DEBUG:teuthology.orchestra.run.smithi197:> grep smithi197.front.sepia.ceph.com /etc/hosts 2021-01-25T03:49:23.121 INFO:teuthology.orchestra.run.smithi197.stdout:172.21.15.197 smithi197.front.sepia.ceph.com smithi197 2021-01-25T03:49:23.122 DEBUG:teuthology.orchestra.run.smithi197:> sudo hostname smithi197 2021-01-25T03:49:23.188 DEBUG:teuthology.orchestra.run.smithi197:> sudo sed -i -e 's/smithi197.front.sepia.ceph.com/smithi197/g' /etc/hosts 2021-01-25T03:49:23.249 DEBUG:teuthology.orchestra.run.smithi197:> sudo sed -i -e 's/172.21.15.197/172.21.15.197/g' /etc/hosts 2021-01-25T03:49:23.306 DEBUG:teuthology.orchestra.run.smithi197:> sudo sed -i -e 's/smithi197.front.sepia.ceph.com/smithi197/g' /etc/hostname 2021-01-25T03:49:23.375 DEBUG:teuthology.orchestra.run.smithi197:> sudo hostname smithi197 2021-01-25T03:49:23.429 INFO:teuthology.provision.fog.smithi197:Deploy complete! 2021-01-25T03:49:23.430 DEBUG:teuthology.parallel:result is None 2021-01-25T03:49:23.751 INFO:teuthology.lock.ops:Checking smithi008.front.sepia.ceph.com 2021-01-25T03:49:23.752 INFO:teuthology.lock.ops:New key found. Updating... 2021-01-25T03:49:23.767 INFO:teuthology.lock.ops:Checking smithi197.front.sepia.ceph.com 2021-01-25T03:49:23.767 INFO:teuthology.lock.ops:New key found. Updating... 2021-01-25T03:49:23.780 INFO:teuthology.lock.ops:Checking smithi125.front.sepia.ceph.com 2021-01-25T03:49:23.781 INFO:teuthology.lock.ops:New key found. Updating... 2021-01-25T03:49:23.795 INFO:teuthology.lock.ops:Checking smithi171.front.sepia.ceph.com 2021-01-25T03:49:23.795 INFO:teuthology.lock.ops:New key found. Updating... 2021-01-25T03:49:23.807 INFO:teuthology.lock.ops:Checking smithi192.front.sepia.ceph.com 2021-01-25T03:49:23.809 INFO:teuthology.lock.ops:New key found. Updating... 2021-01-25T03:49:23.826 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: set os type and version on server 2021-01-25T03:49:23.826 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-25T03:49:23.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:49:23.929 DEBUG:teuthology.orchestra.run.smithi008:> true 2021-01-25T03:49:24.349 DEBUG:teuthology.orchestra.run.smithi008:> uname -m 2021-01-25T03:49:24.432 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64 2021-01-25T03:49:24.432 DEBUG:teuthology.orchestra.run.smithi008:> cat /etc/os-release 2021-01-25T03:49:24.509 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="CentOS Linux" 2021-01-25T03:49:24.510 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="7 (Core)" 2021-01-25T03:49:24.510 INFO:teuthology.orchestra.run.smithi008.stdout:ID="centos" 2021-01-25T03:49:24.510 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE="rhel fedora" 2021-01-25T03:49:24.511 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="7" 2021-01-25T03:49:24.511 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2021-01-25T03:49:24.512 INFO:teuthology.orchestra.run.smithi008.stdout:ANSI_COLOR="0;31" 2021-01-25T03:49:24.512 INFO:teuthology.orchestra.run.smithi008.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2021-01-25T03:49:24.512 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://www.centos.org/" 2021-01-25T03:49:24.513 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-01-25T03:49:24.513 INFO:teuthology.orchestra.run.smithi008.stdout: 2021-01-25T03:49:24.514 INFO:teuthology.orchestra.run.smithi008.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2021-01-25T03:49:24.514 INFO:teuthology.orchestra.run.smithi008.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2021-01-25T03:49:24.514 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-01-25T03:49:24.515 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2021-01-25T03:49:24.515 INFO:teuthology.orchestra.run.smithi008.stdout: 2021-01-25T03:49:24.517 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2021-01-25T03:49:24.533 INFO:teuthology.lock.ops:Updating [smithi197.front.sepia.ceph.com]: set os type and version on server 2021-01-25T03:49:24.534 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-25T03:49:24.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi197.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:49:24.608 DEBUG:teuthology.orchestra.run.smithi197:> true 2021-01-25T03:49:24.803 DEBUG:teuthology.orchestra.run.smithi197:> uname -m 2021-01-25T03:49:24.849 INFO:teuthology.orchestra.run.smithi197.stdout:x86_64 2021-01-25T03:49:24.849 DEBUG:teuthology.orchestra.run.smithi197:> cat /etc/os-release 2021-01-25T03:49:24.899 INFO:teuthology.orchestra.run.smithi197.stdout:NAME="Ubuntu" 2021-01-25T03:49:24.899 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-01-25T03:49:24.900 INFO:teuthology.orchestra.run.smithi197.stdout:ID=ubuntu 2021-01-25T03:49:24.900 INFO:teuthology.orchestra.run.smithi197.stdout:ID_LIKE=debian 2021-01-25T03:49:24.900 INFO:teuthology.orchestra.run.smithi197.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-01-25T03:49:24.901 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION_ID="18.04" 2021-01-25T03:49:24.901 INFO:teuthology.orchestra.run.smithi197.stdout:HOME_URL="https://www.ubuntu.com/" 2021-01-25T03:49:24.901 INFO:teuthology.orchestra.run.smithi197.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-01-25T03:49:24.901 INFO:teuthology.orchestra.run.smithi197.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-01-25T03:49:24.902 INFO:teuthology.orchestra.run.smithi197.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-01-25T03:49:24.902 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION_CODENAME=bionic 2021-01-25T03:49:24.903 INFO:teuthology.orchestra.run.smithi197.stdout:UBUNTU_CODENAME=bionic 2021-01-25T03:49:24.903 INFO:teuthology.lock.ops:Updating smithi197.front.sepia.ceph.com on lock server 2021-01-25T03:49:24.921 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: set os type and version on server 2021-01-25T03:49:24.921 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-25T03:49:24.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:49:24.972 ERROR:paramiko.transport:Unknown exception: The socket has already been closed by another greenlet 2021-01-25T03:49:24.973 ERROR:paramiko.transport:Traceback (most recent call last): 2021-01-25T03:49:24.974 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2055, in run 2021-01-25T03:49:24.974 ERROR:paramiko.transport: ptype, m = self.packetizer.read_message() 2021-01-25T03:49:24.975 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 459, in read_message 2021-01-25T03:49:24.976 ERROR:paramiko.transport: header = self.read_all(self.__block_size_in, check_rekey=True) 2021-01-25T03:49:24.976 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 301, in read_all 2021-01-25T03:49:24.977 ERROR:paramiko.transport: x = self.__socket.recv(n) 2021-01-25T03:49:24.977 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/gevent/_socket3.py", line 382, in recv 2021-01-25T03:49:24.978 ERROR:paramiko.transport: self._wait(self._read_event) 2021-01-25T03:49:24.978 ERROR:paramiko.transport: File "src/gevent/_hub_primitives.py", line 284, in gevent.__hub_primitives.wait_on_socket 2021-01-25T03:49:24.979 ERROR:paramiko.transport: File "src/gevent/_hub_primitives.py", line 288, in gevent.__hub_primitives.wait_on_socket 2021-01-25T03:49:24.979 ERROR:paramiko.transport:gevent.exceptions.ConcurrentObjectUseError: The socket has already been closed by another greenlet 2021-01-25T03:49:24.980 ERROR:paramiko.transport: 2021-01-25T03:49:25.021 DEBUG:teuthology.orchestra.run.smithi125:> true 2021-01-25T03:49:25.429 DEBUG:teuthology.orchestra.run.smithi125:> uname -m 2021-01-25T03:49:25.513 INFO:teuthology.orchestra.run.smithi125.stdout:x86_64 2021-01-25T03:49:25.514 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release 2021-01-25T03:49:25.595 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="CentOS Linux" 2021-01-25T03:49:25.595 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="7 (Core)" 2021-01-25T03:49:25.596 INFO:teuthology.orchestra.run.smithi125.stdout:ID="centos" 2021-01-25T03:49:25.596 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE="rhel fedora" 2021-01-25T03:49:25.596 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="7" 2021-01-25T03:49:25.597 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2021-01-25T03:49:25.597 INFO:teuthology.orchestra.run.smithi125.stdout:ANSI_COLOR="0;31" 2021-01-25T03:49:25.597 INFO:teuthology.orchestra.run.smithi125.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2021-01-25T03:49:25.598 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://www.centos.org/" 2021-01-25T03:49:25.598 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-01-25T03:49:25.598 INFO:teuthology.orchestra.run.smithi125.stdout: 2021-01-25T03:49:25.598 INFO:teuthology.orchestra.run.smithi125.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2021-01-25T03:49:25.599 INFO:teuthology.orchestra.run.smithi125.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2021-01-25T03:49:25.599 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-01-25T03:49:25.599 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2021-01-25T03:49:25.599 INFO:teuthology.orchestra.run.smithi125.stdout: 2021-01-25T03:49:25.600 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server 2021-01-25T03:49:25.615 INFO:teuthology.lock.ops:Updating [smithi171.front.sepia.ceph.com]: set os type and version on server 2021-01-25T03:49:25.615 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-25T03:49:25.616 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:49:25.714 DEBUG:teuthology.orchestra.run.smithi171:> true 2021-01-25T03:49:26.133 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2021-01-25T03:49:26.221 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2021-01-25T03:49:26.222 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2021-01-25T03:49:26.298 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Linux" 2021-01-25T03:49:26.298 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="7 (Core)" 2021-01-25T03:49:26.299 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2021-01-25T03:49:26.299 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2021-01-25T03:49:26.299 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="7" 2021-01-25T03:49:26.300 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2021-01-25T03:49:26.301 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2021-01-25T03:49:26.306 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2021-01-25T03:49:26.306 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://www.centos.org/" 2021-01-25T03:49:26.306 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-01-25T03:49:26.307 INFO:teuthology.orchestra.run.smithi171.stdout: 2021-01-25T03:49:26.315 INFO:teuthology.orchestra.run.smithi171.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2021-01-25T03:49:26.317 INFO:teuthology.orchestra.run.smithi171.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2021-01-25T03:49:26.318 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-01-25T03:49:26.318 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2021-01-25T03:49:26.318 INFO:teuthology.orchestra.run.smithi171.stdout: 2021-01-25T03:49:26.320 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2021-01-25T03:49:26.340 INFO:teuthology.lock.ops:Updating [smithi192.front.sepia.ceph.com]: set os type and version on server 2021-01-25T03:49:26.340 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-01-25T03:49:26.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-01-25T03:49:26.451 DEBUG:teuthology.orchestra.run.smithi192:> true 2021-01-25T03:49:26.857 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2021-01-25T03:49:26.927 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2021-01-25T03:49:26.928 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2021-01-25T03:49:26.998 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Linux" 2021-01-25T03:49:26.999 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="7 (Core)" 2021-01-25T03:49:26.999 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2021-01-25T03:49:26.999 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2021-01-25T03:49:27.000 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="7" 2021-01-25T03:49:27.000 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2021-01-25T03:49:27.001 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2021-01-25T03:49:27.001 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2021-01-25T03:49:27.001 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://www.centos.org/" 2021-01-25T03:49:27.002 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-01-25T03:49:27.002 INFO:teuthology.orchestra.run.smithi192.stdout: 2021-01-25T03:49:27.003 INFO:teuthology.orchestra.run.smithi192.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2021-01-25T03:49:27.003 INFO:teuthology.orchestra.run.smithi192.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2021-01-25T03:49:27.003 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-01-25T03:49:27.004 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2021-01-25T03:49:27.004 INFO:teuthology.orchestra.run.smithi192.stdout: 2021-01-25T03:49:27.005 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2021-01-25T03:49:27.020 INFO:teuthology.task.internal.lock_machines:5 smithi machines locked this try, 5/5 locked so far 2021-01-25T03:49:27.111 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi008.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC445T2uEWXahetzNyjHFxKkS83VOH2XOaePRY87/jhOFchiNGgI5kfiVE7zaOWZAEmCk/wKiiFTZbLtXlykcwpsoCJM+i8+XWlTDNocb9+5rfjUSTmRR0pvzhPdW6OmHza02xSGFgsjlfpThjNfzV5zz7UZuq6zFRQXmYkcugpt3HercASgDtLWjtRsYT8nnWdAK4Dt4pXXDqNHw9Fh7ag0ifjtnFn2QmAUD7+noZDbSVyHtG9GcHZfKLMNpl5uZrSPh5o9YsUekBq6r/ySmCDr7SBBpTTaRRZJ3rvFV6qCwfxFmW0rq/qVDq8qiRB8HyuBdkWEJO7BSN2g9Vx8IAj smithi125.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDtJSg3vKAt+n7/sDXEyLW8iavik5ZBeN4yB+Z5ylTB85/+Ehd6KhOjz7Nu+Z0mTkv6fWrKw3YuxecqSVfU7o4bIKlV3/ZzMOCruHpUd9XN1Nrb0Rif9Fn5O0nIc3f9vul8JzjXAeZ4MzHLGebUmJKCtcLYVwXfSbdu8D1cualzGwaTD/+lv59jDwyqvUZCaPNNZ60tTAMQJrK1U7AWUgPAXlxvDovchGziOx6UP1gFerlT7w5BoOhNiV9Na9F5SrUXzl1l3pnFunJS/0KgVvl/YEfIrxsu3UTkuqgU+xcAEWR9oUbG5grbj349jy9f9pupzzPTwM0XKkbZfKKh/eIJ smithi171.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCrZZgvoUvZrPK4La4ccLoNIZzSI922zR81VVqkkHop22OnzuPE7M0YXxUmpq27TXN49z4jzRqOaIVBUFCeEa3O9xYSo6sH7MdiPWwskr3DHgQUD0F5A1s8b1Szh5Y6vCDtoITL0au8axFHmyUWdK+IBLC66nWRJBH9HcedeP3sye8g/pFpsmVn7iUhnuTSLn8CF4Q4pN34LrwSZeV/0wcjev7jcQ3NuxToBcsrCcLFdcpJIhGVU6SZ/QWG80tXTtoj/vehNURezEoNJx66ZqSxjy2nagMzPkQx7oMzpc3mz2OvMavBl24nyn1g5d3hTltIl3mmuiFT5MXA3HAsBBRr smithi192.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7zyVQT1OBZ8TxJerIiP+tKxLBJpJPNou2jX7srORnJm1xFLPirJtg7Vf7/KI5/A01L6iM6iaQ+z2mbXcPd64yLhDX1zYfJt/a1uRrazJEYk304dh4fr9+kwo4zxzFnRe4SUCS9RRYLomU7j12J9uTDF/TsZzDl7TY8Uvtc3ABAQjKEERhzbuoh3d1l9uyUkYypNhL8KAMSPjpo+lPUo80QEVfhoLo1jFRqIJAg6jwnldp0yMJ59g12Z2xtOkBJLO77fLXUHGKnoI4AsXB5atwdKFR3Gb22aJnpeYHvZ8SoFaHr0yFKORqkMeSZpHIWsj6M+HJlEOp6Q/ophVpae7B smithi197.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDG+GmkqwbO5XmXrsjd+cQ/UXK/cynipYnjywAaiX3M/7z/mzWj2xVJ7eB+8cR0XnsI1ag76MzV7iGM0BYNbMMi2qlBOP9ItxfuzOjtDem/N92O//ekHwg+XSDpcB2p78LSQ5A4zyTSGXJbcj4ew3CjtVvZZcznfKwjAe6JgdHrfYk43uQOzigT5PBmWkkoerW2QRT3NZmejtki0+bv270+jbI00Bvdq9LVvBUtWg05s5c5B+cNt3IalQdo7drdhtRMjt+usLu9OD+qcXRxnxVxMN+pdtVkenlUfAaEQTmrDe8PSWChVi9xG34bazqkZO1RkBi24Clt3SFTKwWgr7dP 2021-01-25T03:49:27.111 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-01-25T03:49:27.184 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-01-25T03:49:27.188 INFO:teuthology.task.internal:Saving configuration 2021-01-25T03:49:27.275 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-01-25T03:49:27.288 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-01-25T03:49:27.306 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': False, 'locked_since': None, 'description': None, 'locked_by': None, 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKbf7b9nsUUyxRmCK7pB6mDhtd7jMz8VTWN3OFOyOVKuGVKIMYKaUjdEJ1JO5bTrN0z1oBVNnaFMPdw/d2aFZpU=', 'os_version': '7', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi008.front.sepia.ceph.com'} 2021-01-25T03:49:27.307 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 91, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 70, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/check_lock.py", line 29, in check_lock 'machine {name} is not locked'.format(name=machine) AssertionError: machine smithi008.front.sepia.ceph.com is not locked 2021-01-25T03:49:27.396 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5213951eb54c4f0db7ec41c65167ea2a Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 91, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 70, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/check_lock.py", line 29, in check_lock 'machine {name} is not locked'.format(name=machine) AssertionError: machine smithi008.front.sepia.ceph.com is not locked 2021-01-25T03:49:27.398 DEBUG:teuthology.run_tasks:Unwinding manager internal.lock_machines 2021-01-25T03:49:27.419 INFO:teuthology.nuke:Checking targets against current locks 2021-01-25T03:49:27.540 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/scripts/run.py", line 38, in main teuthology.run.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run.py", line 411, in main report_outcome(config, archive, fake_ctx.summary, fake_ctx) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run.py", line 268, in report_outcome nuke(fake_ctx, fake_ctx.lock) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/nuke/__init__.py", line 250, in nuke if ctx.name not in lock['description']: TypeError: argument of type 'NoneType' is not iterable