2020-03-27T18:08:48.865 INFO:root:teuthology version: 1.0.0-b3a6b1a 2020-03-27T18:08:48.866 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.ERrDUh.tmp -- --verbose --lock --description rados:thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/nautilus-v1only.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/off.yaml distro$/{centos_7.6.yaml} msgr-failures/fastclose.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} --name sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi --block --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi/4895635 2020-03-27T18:08:48.902 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-03-27T18:08:48.961 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi/4895635 branch: revert-dec description: rados:thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-install/nautilus-v1only.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/off.yaml distro$/{centos_7.6.yaml} msgr-failures/fastclose.yaml rados.yaml thrashers/default.yaml thrashosds-health.yaml workloads/snaps-few-objects.yaml} email: null first_in_suite: false job_id: '4895635' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 30 - volumes: count: 4 size: 10 os_type: centos os_version: '7.6' overrides: admin_socket: branch: revert-dec ceph: conf: global: mon client directed command retry: 5 ms bind msgr2: false ms inject socket failures: 2500 ms tcp read timeout: 5 ms type: async osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 osd backoff on degraded: true osd backoff on peering: true osd debug reject backfill probability: 0.3 osd debug verify cached snaps: true osd debug verify missing on start: true osd max backfills: 3 osd max markdown count: 1000 osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(MON_DOWN\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request mon_bind_msgr2: false sha1: df873c1654814c23dcb68f897a012936832bfab9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: df873c1654814c23dcb68f897a012936832bfab9 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: min_in: 4 workunit: branch: revert-dec sha1: df873c1654814c23dcb68f897a012936832bfab9 owner: scheduled_sage@teuthology priority: 50 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - - mon.c - osd.8 - osd.9 - osd.10 - osd.11 - client.2 sha1: df873c1654814c23dcb68f897a012936832bfab9 suite: rados:thrash-old-clients suite_branch: revert-dec suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_revert-dec/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: df873c1654814c23dcb68f897a012936832bfab9 tasks: - install: branch: nautilus exclude_packages: - cephadm - ceph-mgr-cephadm - ceph-immutable-object-cache - python3-rados - python3-rgw - python3-rbd - python3-cephfs extra_packages: - python-rados - python-rgw - python-rbd - python-cephfs - cephadm: null - thrashosds: chance_pgnum_grow: 1 chance_pgpnum_fix: 1 timeout: 1200 - rados: clients: - client.2 objects: 50 op_weights: copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.2750 2020-03-27T18:08:48.961 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_revert-dec/qa; will attempt to use it 2020-03-27T18:08:48.962 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_revert-dec/qa/tasks 2020-03-27T18:08:48.962 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-03-27T18:08:48.964 INFO:teuthology.task.internal:Checking packages... 2020-03-27T18:08:48.976 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash 'df873c1654814c23dcb68f897a012936832bfab9' 2020-03-27T18:08:48.977 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-03-27T18:08:48.977 INFO:teuthology.packaging:ref: None 2020-03-27T18:08:48.977 INFO:teuthology.packaging:tag: None 2020-03-27T18:08:48.977 INFO:teuthology.packaging:branch: revert-dec 2020-03-27T18:08:48.977 INFO:teuthology.packaging:sha1: df873c1654814c23dcb68f897a012936832bfab9 2020-03-27T18:08:48.977 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&ref=revert-dec 2020-03-27T18:08:49.216 INFO:teuthology.task.internal:Found packages for ceph version 15.2.0-377.gdf873c1 2020-03-27T18:08:49.217 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-03-27T18:08:49.227 INFO:teuthology.task.internal:no buildpackages task found 2020-03-27T18:08:49.228 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-03-27T18:08:49.242 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-03-27T18:08:49.242 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-03-27T18:08:49.300 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:08:59.318 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:09:09.336 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:09:19.354 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:09:29.376 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:09:39.394 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:09:49.409 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:09:59.420 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:10:09.446 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 0)... 2020-03-27T18:10:19.460 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:10:29.481 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:10:39.500 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:10:49.522 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:10:59.562 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:11:09.579 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:11:19.597 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:11:29.628 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 4)... 2020-03-27T18:11:39.645 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:11:49.666 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:11:59.677 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:12:09.694 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:12:19.709 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:12:29.724 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:12:39.741 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:12:49.760 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:12:59.798 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:13:09.818 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:13:19.834 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:13:29.849 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:13:39.880 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:13:49.896 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:13:59.917 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 4)... 2020-03-27T18:14:09.934 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:14:19.950 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:14:29.965 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:14:39.980 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:14:49.990 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:15:00.005 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:15:10.021 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:15:20.036 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:15:30.056 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:15:40.081 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:15:50.095 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:16:00.112 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:16:10.153 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:16:20.171 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:16:30.189 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:16:40.207 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:16:50.225 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:17:00.241 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:17:10.256 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:17:20.272 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:17:30.284 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:17:40.300 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:17:50.316 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:18:00.332 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:18:10.349 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:18:20.365 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:18:30.383 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:18:40.396 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:18:50.412 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:19:00.434 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:19:10.448 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:19:20.465 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:19:30.482 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:19:40.501 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:19:50.517 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:20:00.533 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:20:10.544 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:20:20.561 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:20:30.587 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:20:40.607 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:20:50.629 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:21:00.650 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:21:10.663 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:21:20.682 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:21:30.699 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:21:40.716 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:21:50.737 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:22:00.758 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:22:10.775 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:22:20.792 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:22:30.801 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:22:40.817 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:22:50.832 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:23:00.859 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:23:10.881 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:23:20.896 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:23:30.917 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:23:40.932 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:23:50.949 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:24:00.965 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:24:10.983 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:24:21.003 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 4)... 2020-03-27T18:24:31.023 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:24:41.055 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:24:51.071 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:25:01.091 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:25:11.112 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:25:21.135 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:25:31.152 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:25:41.168 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:25:51.186 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:26:01.202 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:26:11.216 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:26:21.250 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 4)... 2020-03-27T18:26:31.265 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:26:41.283 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:26:51.296 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:27:01.313 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:27:11.329 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:27:21.346 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:27:31.354 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:27:41.372 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:27:51.388 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:28:01.412 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:28:11.429 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:28:21.446 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:28:31.461 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:28:41.478 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:28:51.507 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:29:01.525 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:29:11.543 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:29:21.560 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:29:31.574 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:29:41.588 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:29:51.604 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:30:01.622 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:30:11.645 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:30:21.661 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:30:31.699 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:30:41.717 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:30:51.733 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:31:01.749 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:31:11.766 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:31:21.780 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:31:31.798 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:31:41.816 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:31:51.833 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:32:01.846 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:32:11.860 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:32:21.877 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:32:31.904 DEBUG:teuthology.lock.ops:lock_many request: {'count': 3, 'locked_by': 'scheduled_sage@teuthology', 'description': '/home/teuthworker/archive/sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi/4895635', 'machine_type': 'smithi'} 2020-03-27T18:32:31.921 DEBUG:teuthology.lock.ops:locked ubuntu@smithi050.front.sepia.ceph.com, ubuntu@smithi164.front.sepia.ceph.com, ubuntu@smithi155.front.sepia.ceph.com 2020-03-27T18:32:32.614 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2020-03-27T18:32:32.874 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of centos 7.6 2020-03-27T18:32:32.882 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of centos 7.6 2020-03-27T18:32:32.885 INFO:teuthology.provision.fog.smithi155:Scheduling deploy of centos 7.6 2020-03-27T18:32:33.279 INFO:teuthology.orchestra.console:Power off smithi050 2020-03-27T18:32:33.280 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-03-27T18:32:33.300 INFO:teuthology.orchestra.console:Power off smithi155 2020-03-27T18:32:33.301 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-03-27T18:32:33.318 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:33.356 INFO:teuthology.orchestra.console:Power off smithi164 2020-03-27T18:32:33.357 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-03-27T18:32:33.366 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:33.379 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:33.494 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2020-03-27T18:32:33.595 INFO:teuthology.orchestra.console:Power on smithi155 2020-03-27T18:32:33.595 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-03-27T18:32:33.616 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:37.377 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:37.401 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:37.645 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:41.497 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:41.510 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:41.708 INFO:teuthology.orchestra.console:Power off for smithi050 completed 2020-03-27T18:32:41.726 INFO:teuthology.orchestra.console:Power off for smithi164 completed 2020-03-27T18:32:41.765 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:41.809 INFO:teuthology.orchestra.console:Power on smithi050 2020-03-27T18:32:41.810 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-03-27T18:32:41.828 INFO:teuthology.orchestra.console:Power on smithi164 2020-03-27T18:32:41.828 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-03-27T18:32:41.849 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:41.859 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:41.978 INFO:teuthology.orchestra.console:Power on for smithi155 completed 2020-03-27T18:32:42.078 INFO:teuthology.provision.fog.smithi155:Waiting for deploy to finish 2020-03-27T18:32:45.873 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:45.884 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:49.990 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:49.999 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:32:50.202 INFO:teuthology.orchestra.console:Power on for smithi050 completed 2020-03-27T18:32:50.216 INFO:teuthology.orchestra.console:Power on for smithi164 completed 2020-03-27T18:32:50.304 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish 2020-03-27T18:32:50.321 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish 2020-03-27T18:35:21.464 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi050.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:30.534 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi050.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:36.543 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi164.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:39.597 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi050.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:43.356 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi155.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:45.610 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi164.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:48.677 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi050.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:52.426 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi155.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:54.662 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi164.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:57.733 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi050.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:36:01.478 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi155.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:36:03.718 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi164.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:36:06.789 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi050.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:36:10.534 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi155.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:36:12.774 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi164.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:36:15.846 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi050.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:36:19.601 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi155.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:36:21.838 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi164.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:36:28.680 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi155.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:36:30.920 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi164.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:36:37.737 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi155.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:36:39.978 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi164.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:36:46.793 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi155.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:36:47.633 INFO:teuthology.orchestra.run.smithi050:> hostname 2020-03-27T18:36:48.142 INFO:teuthology.orchestra.run.smithi050.stdout:smithi125 2020-03-27T18:36:48.142 INFO:teuthology.orchestra.run.smithi050:> grep smithi125 /etc/hosts 2020-03-27T18:36:48.220 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.125 smithi125.front.sepia.ceph.com smithi125 2020-03-27T18:36:48.220 INFO:teuthology.orchestra.run.smithi050:> sudo hostname smithi050 2020-03-27T18:36:48.331 INFO:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi125/smithi050/g' /etc/hosts 2020-03-27T18:36:48.397 INFO:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/172.21.15.125/172.21.15.50/g' /etc/hosts 2020-03-27T18:36:48.455 INFO:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi125/smithi050/g' /etc/hostname 2020-03-27T18:36:48.514 INFO:teuthology.orchestra.run.smithi050:> sudo hostname smithi050 2020-03-27T18:36:48.569 INFO:teuthology.provision.fog.smithi050:Deploy complete! 2020-03-27T18:36:48.570 DEBUG:teuthology.parallel:result is None 2020-03-27T18:36:55.854 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi155.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:37:17.514 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi164.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:37:17.793 INFO:teuthology.orchestra.run.smithi164:> hostname 2020-03-27T18:37:18.267 INFO:teuthology.orchestra.run.smithi164.stdout:smithi125 2020-03-27T18:37:18.267 INFO:teuthology.orchestra.run.smithi164:> grep smithi125 /etc/hosts 2020-03-27T18:37:18.321 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.125 smithi125.front.sepia.ceph.com smithi125 2020-03-27T18:37:18.321 INFO:teuthology.orchestra.run.smithi164:> sudo hostname smithi164 2020-03-27T18:37:18.416 INFO:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi125/smithi164/g' /etc/hosts 2020-03-27T18:37:18.476 INFO:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/172.21.15.125/172.21.15.164/g' /etc/hosts 2020-03-27T18:37:18.538 INFO:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi125/smithi164/g' /etc/hostname 2020-03-27T18:37:18.595 INFO:teuthology.orchestra.run.smithi164:> sudo hostname smithi164 2020-03-27T18:37:18.648 INFO:teuthology.provision.fog.smithi164:Deploy complete! 2020-03-27T18:37:18.650 DEBUG:teuthology.parallel:result is None 2020-03-27T18:38:01.894 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi155.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:38:02.119 INFO:teuthology.orchestra.run.smithi155:> hostname 2020-03-27T18:38:02.682 INFO:teuthology.orchestra.run.smithi155.stdout:smithi155.front.sepia.ceph.com 2020-03-27T18:38:02.682 INFO:teuthology.orchestra.run.smithi155:> grep smithi155.front.sepia.ceph.com /etc/hosts 2020-03-27T18:38:02.768 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.15.155 smithi155.front.sepia.ceph.com smithi155 2020-03-27T18:38:02.769 INFO:teuthology.orchestra.run.smithi155:> sudo hostname smithi155 2020-03-27T18:38:02.925 INFO:teuthology.orchestra.run.smithi155:> sudo sed -i -e 's/smithi155.front.sepia.ceph.com/smithi155/g' /etc/hosts 2020-03-27T18:38:03.050 INFO:teuthology.orchestra.run.smithi155:> sudo sed -i -e 's/172.21.15.155/172.21.15.155/g' /etc/hosts 2020-03-27T18:38:03.149 INFO:teuthology.orchestra.run.smithi155:> sudo sed -i -e 's/smithi155.front.sepia.ceph.com/smithi155/g' /etc/hostname 2020-03-27T18:38:03.241 INFO:teuthology.orchestra.run.smithi155:> sudo hostname smithi155 2020-03-27T18:38:03.314 INFO:teuthology.provision.fog.smithi155:Deploy complete! 2020-03-27T18:38:03.316 DEBUG:teuthology.parallel:result is None 2020-03-27T18:38:03.550 INFO:teuthology.lock.ops:Checking smithi155.front.sepia.ceph.com 2020-03-27T18:38:03.551 INFO:teuthology.lock.ops:New key found. Updating... 2020-03-27T18:38:03.572 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com 2020-03-27T18:38:03.572 INFO:teuthology.lock.ops:New key found. Updating... 2020-03-27T18:38:03.588 INFO:teuthology.lock.ops:Checking smithi050.front.sepia.ceph.com 2020-03-27T18:38:03.588 INFO:teuthology.lock.ops:New key found. Updating... 2020-03-27T18:38:03.604 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: set os type and version on server 2020-03-27T18:38:03.604 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-03-27T18:38:03.605 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi155.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:38:03.712 INFO:teuthology.orchestra.run.smithi155:> true 2020-03-27T18:38:04.088 INFO:teuthology.orchestra.run.smithi155:> uname -m 2020-03-27T18:38:04.149 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2020-03-27T18:38:04.149 INFO:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2020-03-27T18:38:04.206 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Linux" 2020-03-27T18:38:04.206 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="7 (Core)" 2020-03-27T18:38:04.206 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2020-03-27T18:38:04.206 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2020-03-27T18:38:04.207 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="7" 2020-03-27T18:38:04.207 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-03-27T18:38:04.207 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2020-03-27T18:38:04.207 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-03-27T18:38:04.208 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.centos.org/" 2020-03-27T18:38:04.208 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-03-27T18:38:04.208 INFO:teuthology.orchestra.run.smithi155.stdout: 2020-03-27T18:38:04.208 INFO:teuthology.orchestra.run.smithi155.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-03-27T18:38:04.208 INFO:teuthology.orchestra.run.smithi155.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-03-27T18:38:04.208 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-03-27T18:38:04.209 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-03-27T18:38:04.209 INFO:teuthology.orchestra.run.smithi155.stdout: 2020-03-27T18:38:04.210 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2020-03-27T18:38:04.226 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server 2020-03-27T18:38:04.226 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-03-27T18:38:04.226 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi164.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:38:04.325 INFO:teuthology.orchestra.run.smithi164:> true 2020-03-27T18:38:04.690 INFO:teuthology.orchestra.run.smithi164:> uname -m 2020-03-27T18:38:04.750 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2020-03-27T18:38:04.750 INFO:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2020-03-27T18:38:04.805 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Linux" 2020-03-27T18:38:04.805 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="7 (Core)" 2020-03-27T18:38:04.805 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2020-03-27T18:38:04.805 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2020-03-27T18:38:04.805 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="7" 2020-03-27T18:38:04.806 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-03-27T18:38:04.806 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2020-03-27T18:38:04.806 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-03-27T18:38:04.806 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.centos.org/" 2020-03-27T18:38:04.806 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-03-27T18:38:04.806 INFO:teuthology.orchestra.run.smithi164.stdout: 2020-03-27T18:38:04.806 INFO:teuthology.orchestra.run.smithi164.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-03-27T18:38:04.806 INFO:teuthology.orchestra.run.smithi164.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-03-27T18:38:04.807 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-03-27T18:38:04.807 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-03-27T18:38:04.807 INFO:teuthology.orchestra.run.smithi164.stdout: 2020-03-27T18:38:04.807 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2020-03-27T18:38:04.821 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: set os type and version on server 2020-03-27T18:38:04.822 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-03-27T18:38:04.822 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi050.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:38:04.932 INFO:teuthology.orchestra.run.smithi050:> true 2020-03-27T18:38:05.341 INFO:teuthology.orchestra.run.smithi050:> uname -m 2020-03-27T18:38:05.417 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2020-03-27T18:38:05.418 INFO:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2020-03-27T18:38:05.502 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Linux" 2020-03-27T18:38:05.502 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="7 (Core)" 2020-03-27T18:38:05.502 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2020-03-27T18:38:05.503 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2020-03-27T18:38:05.503 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="7" 2020-03-27T18:38:05.503 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-03-27T18:38:05.503 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2020-03-27T18:38:05.503 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-03-27T18:38:05.503 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://www.centos.org/" 2020-03-27T18:38:05.503 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-03-27T18:38:05.504 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-03-27T18:38:05.504 INFO:teuthology.orchestra.run.smithi050.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-03-27T18:38:05.504 INFO:teuthology.orchestra.run.smithi050.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-03-27T18:38:05.504 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-03-27T18:38:05.504 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-03-27T18:38:05.504 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-03-27T18:38:05.505 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2020-03-27T18:38:05.520 INFO:teuthology.task.internal.lock_machines:3 smithi machines locked this try, 3/3 locked so far 2020-03-27T18:38:05.570 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi050.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzt+5zpb+GhswBDcOf1AP3HT7ZTtHd+SeYzs5iu3HI7C3bGso+Hj6HGof9nibWeuEXkXoJq1Fhtlc+xhAD7fRL4KrJNnZ0Sc9vjHdEPsMbrRd08xojhV1wkBEw+ehfesIExlIFmI/uwXmwx8IzxpEky3wdgElLntf+jF4/kjExPE6K9JY0QjVl/VxthtwXWAqp4FMyUkgmb0nlRCIJ9/7pfabO6RMsQhrd/3Rn+FKtZ7uNIOolBgvR/cOi334QYL+GvBNIf0RgEq2Nj1AWmkibWFTylLUeDSVkpUEJMUtIYJnHi7EQcVXmfxhBayAcPjh77y7aht4h9R65tCODXIu9 smithi155.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj2dvw5ybDfoRlzRuYMoH/Uw66ET6adrkNOQn5XZxU7EPP0+nG338QTCfy9nw1kcwPJ0GdQv0oFQFtNeCcm9fyNAr4/cRcSwWaS6Kn4GJEX5keNzrsbpG3J0i7gNSHdOzbftyova6OiHT+v7zo3rB9fGLktrQL020LNwZQPSxm7ZLgjo6G1o07Pe5IGkBoYYaPYfOayMOzC6sB+TZPpZPE83BvxlR7jxJ04p/xK8ZZEcsJ9IRSP1vdplaFuyLeRZ0tplEXvKRghVBy6hfIh0Ix3zttLZcMvELwsvEEUsOGa71aOZ+PT/lY4RgBbRFQsYul3aim8bvk6xmlBGErNVSp smithi164.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFmjUHMiIqND3pqlB/Zao+cLbdfGZKwlmptgM4jczyP4lewAOAGdXb6TehmW0fQaqCP2o2YYvhUIsJKnf6KUBPJutQEOWYZBavvPA+mFAwMU9zw6MUz1ATnXRnTBDqsJlaecXrBTA71sYxBuE/aNoerRyHfjquwDFmVlFchf+Pvb2sVbBLFlbvgsDYRS/copC3yZJuyM2A5NS5SQfwdJCekdBrzxphEPJK2x21AVIl6A7s7/ipx/8+/0g50lj69XG0SIg1xEcfUUyPTvG6J0uxukpRmOfpbLtOciqvqcHTjgFuqH5nvOzXXwdoxrlMZ2K5nnJ7tkt5l6BzCS5kjXEB 2020-03-27T18:38:05.570 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-03-27T18:38:05.624 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-03-27T18:38:05.630 INFO:teuthology.task.internal:Saving configuration 2020-03-27T18:38:05.649 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-03-27T18:38:05.659 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-03-27T18:38:05.679 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2020-03-27 18:32:31.910907', u'locked_by': u'scheduled_sage@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi164.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEtK6MzaII6SsYRUrmGfdeYaYdPnmd+tTqq2/rCfGBHHEKKj0GjAqQp4EdfHsJd87k6ZNAPSdNTLml0JRofQ9pg=', u'description': u'/home/teuthworker/archive/sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi/4895635'} 2020-03-27T18:38:05.698 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2020-03-27 18:32:31.910887', u'locked_by': u'scheduled_sage@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi155.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDeepx38SAYNOqp9lqj3MvWB+gwFULWRZv72R6FYOShnnIFJyIi5hzDqP61vZzuyy6Du5CLRuYgFIuzgZzZBIUc=', u'description': u'/home/teuthworker/archive/sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi/4895635'} 2020-03-27T18:38:05.711 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2020-03-27 18:32:31.910922', u'locked_by': u'scheduled_sage@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi050.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPhXIakNX/PqK8CfUCMfsRsEmdqH6q5xHx5MR1eWZIkGVDTAzbJfZgPd/lFdc8Wp/AlJWW+B/DmT2tk24u1ycdU=', u'description': u'/home/teuthworker/archive/sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi/4895635'} 2020-03-27T18:38:05.712 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-03-27T18:38:05.719 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2020-03-27T18:38:05.719 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2020-03-27T18:38:05.719 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.c', 'osd.8', 'osd.9', 'osd.10', 'osd.11', 'client.2'] 2020-03-27T18:38:05.720 INFO:teuthology.run_tasks:Running task console_log... 2020-03-27T18:38:05.825 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2020-03-27T18:38:05.826 INFO:teuthology.run_tasks:Running task internal.connect... 2020-03-27T18:38:05.835 INFO:teuthology.task.internal:Opening connections... 2020-03-27T18:38:05.835 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2020-03-27T18:38:05.836 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi164.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:38:05.940 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2020-03-27T18:38:05.941 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi155.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:38:06.085 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2020-03-27T18:38:06.085 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi050.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:38:06.190 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-03-27T18:38:06.200 INFO:teuthology.orchestra.run.smithi164:> uname -m 2020-03-27T18:38:06.317 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2020-03-27T18:38:06.318 INFO:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2020-03-27T18:38:06.398 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Linux" 2020-03-27T18:38:06.399 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="7 (Core)" 2020-03-27T18:38:06.399 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2020-03-27T18:38:06.399 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2020-03-27T18:38:06.399 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="7" 2020-03-27T18:38:06.399 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-03-27T18:38:06.400 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2020-03-27T18:38:06.400 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-03-27T18:38:06.400 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.centos.org/" 2020-03-27T18:38:06.400 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-03-27T18:38:06.400 INFO:teuthology.orchestra.run.smithi164.stdout: 2020-03-27T18:38:06.400 INFO:teuthology.orchestra.run.smithi164.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-03-27T18:38:06.401 INFO:teuthology.orchestra.run.smithi164.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-03-27T18:38:06.401 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-03-27T18:38:06.401 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-03-27T18:38:06.401 INFO:teuthology.orchestra.run.smithi164.stdout: 2020-03-27T18:38:06.401 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2020-03-27T18:38:06.416 INFO:teuthology.orchestra.run.smithi155:> uname -m 2020-03-27T18:38:06.542 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2020-03-27T18:38:06.542 INFO:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2020-03-27T18:38:06.614 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Linux" 2020-03-27T18:38:06.614 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="7 (Core)" 2020-03-27T18:38:06.614 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2020-03-27T18:38:06.615 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2020-03-27T18:38:06.615 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="7" 2020-03-27T18:38:06.615 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-03-27T18:38:06.615 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2020-03-27T18:38:06.615 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-03-27T18:38:06.616 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.centos.org/" 2020-03-27T18:38:06.616 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-03-27T18:38:06.616 INFO:teuthology.orchestra.run.smithi155.stdout: 2020-03-27T18:38:06.616 INFO:teuthology.orchestra.run.smithi155.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-03-27T18:38:06.616 INFO:teuthology.orchestra.run.smithi155.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-03-27T18:38:06.617 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-03-27T18:38:06.617 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-03-27T18:38:06.617 INFO:teuthology.orchestra.run.smithi155.stdout: 2020-03-27T18:38:06.617 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2020-03-27T18:38:06.631 INFO:teuthology.orchestra.run.smithi050:> uname -m 2020-03-27T18:38:06.755 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2020-03-27T18:38:06.756 INFO:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2020-03-27T18:38:06.842 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Linux" 2020-03-27T18:38:06.843 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="7 (Core)" 2020-03-27T18:38:06.843 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2020-03-27T18:38:06.843 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2020-03-27T18:38:06.843 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="7" 2020-03-27T18:38:06.843 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-03-27T18:38:06.843 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2020-03-27T18:38:06.843 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-03-27T18:38:06.844 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://www.centos.org/" 2020-03-27T18:38:06.844 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-03-27T18:38:06.844 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-03-27T18:38:06.844 INFO:teuthology.orchestra.run.smithi050.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-03-27T18:38:06.844 INFO:teuthology.orchestra.run.smithi050.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-03-27T18:38:06.844 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-03-27T18:38:06.844 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-03-27T18:38:06.845 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-03-27T18:38:06.845 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2020-03-27T18:38:06.860 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-03-27T18:38:06.928 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-03-27T18:38:06.938 INFO:teuthology.task.internal:Checking for old test directory... 2020-03-27T18:38:06.946 INFO:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2020-03-27T18:38:06.952 INFO:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2020-03-27T18:38:06.956 INFO:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2020-03-27T18:38:07.044 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-03-27T18:38:07.051 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-03-27T18:38:07.052 INFO:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2020-03-27T18:38:07.073 INFO:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2020-03-27T18:38:07.084 INFO:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2020-03-27T18:38:07.197 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-03-27T18:38:07.262 INFO:teuthology.run_tasks:Running task kernel... 2020-03-27T18:38:07.278 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'osd.8': {'sha1': 'distro', 'kdb': True}, 'mon.c': {'sha1': 'distro', 'kdb': True}, 'client.0': {'sha1': 'distro', 'kdb': True}, 'osd.1': {'sha1': 'distro', 'kdb': True}, 'osd.0': {'sha1': 'distro', 'kdb': True}, 'osd.3': {'sha1': 'distro', 'kdb': True}, 'osd.2': {'sha1': 'distro', 'kdb': True}, 'osd.5': {'sha1': 'distro', 'kdb': True}, 'osd.4': {'sha1': 'distro', 'kdb': True}, 'osd.7': {'sha1': 'distro', 'kdb': True}, 'osd.6': {'sha1': 'distro', 'kdb': True}, 'osd.11': {'sha1': 'distro', 'kdb': True}, 'osd.10': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}, 'mgr.y': {'sha1': 'distro', 'kdb': True}, 'mgr.x': {'sha1': 'distro', 'kdb': True}, 'osd.9': {'sha1': 'distro', 'kdb': True}, 'client.2': {'sha1': 'distro', 'kdb': True}, 'client.1': {'sha1': 'distro', 'kdb': True}} 2020-03-27T18:38:07.278 INFO:teuthology.task.kernel:config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.c': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}}, timeout 300 2020-03-27T18:38:07.279 INFO:teuthology.orchestra.run.smithi164:> uname -r 2020-03-27T18:38:07.368 INFO:teuthology.orchestra.run.smithi164.stdout:3.10.0-1062.12.1.el7.x86_64 2020-03-27T18:38:07.369 INFO:teuthology.task.kernel:Running kernel on smithi164: 3.10.0-1062.12.1.el7.x86_64 2020-03-27T18:38:07.369 INFO:teuthology.orchestra.run.smithi164:> sudo yum install -y kernel 2020-03-27T18:38:08.242 INFO:teuthology.orchestra.run.smithi164.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-03-27T18:38:08.355 INFO:teuthology.orchestra.run.smithi164.stdout:Determining fastest mirrors 2020-03-27T18:38:11.721 INFO:teuthology.orchestra.run.smithi164.stdout: * base: mirror.net.cen.ct.gov 2020-03-27T18:38:11.723 INFO:teuthology.orchestra.run.smithi164.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-03-27T18:38:11.724 INFO:teuthology.orchestra.run.smithi164.stdout: * extras: packages.oit.ncsu.edu 2020-03-27T18:38:11.724 INFO:teuthology.orchestra.run.smithi164.stdout: * updates: mirror.siena.edu 2020-03-27T18:38:15.175 INFO:teuthology.orchestra.run.smithi164.stdout:Resolving Dependencies 2020-03-27T18:38:15.176 INFO:teuthology.orchestra.run.smithi164.stdout:--> Running transaction check 2020-03-27T18:38:15.176 INFO:teuthology.orchestra.run.smithi164.stdout:---> Package kernel.x86_64 0:3.10.0-1062.18.1.el7 will be installed 2020-03-27T18:38:16.424 INFO:teuthology.orchestra.run.smithi164.stdout:--> Finished Dependency Resolution 2020-03-27T18:38:17.231 INFO:teuthology.orchestra.run.smithi164.stdout: 2020-03-27T18:38:17.231 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies Resolved 2020-03-27T18:38:17.232 INFO:teuthology.orchestra.run.smithi164.stdout: 2020-03-27T18:38:17.233 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2020-03-27T18:38:17.233 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2020-03-27T18:38:17.241 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2020-03-27T18:38:17.241 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2020-03-27T18:38:17.241 INFO:teuthology.orchestra.run.smithi164.stdout: kernel x86_64 3.10.0-1062.18.1.el7 updates 50 M 2020-03-27T18:38:17.241 INFO:teuthology.orchestra.run.smithi164.stdout: 2020-03-27T18:38:17.241 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2020-03-27T18:38:17.241 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2020-03-27T18:38:17.242 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2020-03-27T18:38:17.242 INFO:teuthology.orchestra.run.smithi164.stdout: 2020-03-27T18:38:17.242 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 50 M 2020-03-27T18:38:17.243 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 64 M 2020-03-27T18:38:17.243 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading packages: 2020-03-27T18:38:17.243 INFO:teuthology.orchestra.run.smithi164.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2020-03-27T18:38:18.139 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2020-03-27T18:38:18.336 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2020-03-27T18:38:19.308 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded 2020-03-27T18:38:19.311 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2020-03-27T18:38:54.232 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-3.10.0-1062.18.1.el7.x86_64 1/1 2020-03-27T18:38:54.735 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-3.10.0-1062.18.1.el7.x86_64 1/1 2020-03-27T18:38:54.735 INFO:teuthology.orchestra.run.smithi164.stdout: 2020-03-27T18:38:54.735 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2020-03-27T18:38:54.735 INFO:teuthology.orchestra.run.smithi164.stdout: kernel.x86_64 0:3.10.0-1062.18.1.el7 2020-03-27T18:38:54.735 INFO:teuthology.orchestra.run.smithi164.stdout: 2020-03-27T18:38:54.736 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2020-03-27T18:38:54.787 INFO:teuthology.orchestra.run.smithi164:> rpm -q kernel --last | head -n 1 2020-03-27T18:38:55.181 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-3.10.0-1062.18.1.el7.x86_64 Fri 27 Mar 2020 06:38:22 PM UTC 2020-03-27T18:38:55.184 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:38:55.184 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.12.1.el7.x86_64 Expected: 3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:38:55.184 INFO:teuthology.orchestra.run.smithi050:> uname -r 2020-03-27T18:38:55.281 INFO:teuthology.orchestra.run.smithi050.stdout:3.10.0-1062.12.1.el7.x86_64 2020-03-27T18:38:55.281 INFO:teuthology.task.kernel:Running kernel on smithi050: 3.10.0-1062.12.1.el7.x86_64 2020-03-27T18:38:55.281 INFO:teuthology.orchestra.run.smithi050:> sudo yum install -y kernel 2020-03-27T18:38:56.229 INFO:teuthology.orchestra.run.smithi050.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-03-27T18:38:56.332 INFO:teuthology.orchestra.run.smithi050.stdout:Determining fastest mirrors 2020-03-27T18:38:59.927 INFO:teuthology.orchestra.run.smithi050.stdout: * base: repos-va.psychz.net 2020-03-27T18:38:59.935 INFO:teuthology.orchestra.run.smithi050.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-03-27T18:38:59.935 INFO:teuthology.orchestra.run.smithi050.stdout: * extras: packages.oit.ncsu.edu 2020-03-27T18:38:59.937 INFO:teuthology.orchestra.run.smithi050.stdout: * updates: mirror.siena.edu 2020-03-27T18:39:03.340 INFO:teuthology.orchestra.run.smithi050.stdout:Resolving Dependencies 2020-03-27T18:39:03.341 INFO:teuthology.orchestra.run.smithi050.stdout:--> Running transaction check 2020-03-27T18:39:03.342 INFO:teuthology.orchestra.run.smithi050.stdout:---> Package kernel.x86_64 0:3.10.0-1062.18.1.el7 will be installed 2020-03-27T18:39:04.544 INFO:teuthology.orchestra.run.smithi050.stdout:--> Finished Dependency Resolution 2020-03-27T18:39:05.375 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-03-27T18:39:05.376 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies Resolved 2020-03-27T18:39:05.376 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-03-27T18:39:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-03-27T18:39:05.377 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2020-03-27T18:39:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-03-27T18:39:05.377 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2020-03-27T18:39:05.377 INFO:teuthology.orchestra.run.smithi050.stdout: kernel x86_64 3.10.0-1062.18.1.el7 updates 50 M 2020-03-27T18:39:05.377 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-03-27T18:39:05.378 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2020-03-27T18:39:05.378 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2020-03-27T18:39:05.378 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2020-03-27T18:39:05.378 INFO:teuthology.orchestra.run.smithi050.stdout: 2020-03-27T18:39:05.379 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 50 M 2020-03-27T18:39:05.379 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 64 M 2020-03-27T18:39:05.379 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading packages: 2020-03-27T18:39:05.387 INFO:teuthology.orchestra.run.smithi050.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2020-03-27T18:39:06.373 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2020-03-27T18:39:06.580 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2020-03-27T18:39:07.426 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded 2020-03-27T18:39:07.427 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2020-03-27T18:39:19.949 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2020-03-27T18:39:19.949 DEBUG:teuthology.task.console_log:Killing console logger for smithi155 2020-03-27T18:39:19.950 DEBUG:teuthology.task.console_log:Killing console logger for smithi164 2020-03-27T18:39:19.950 DEBUG:teuthology.task.console_log:Killing console logger for smithi050 2020-03-27T18:39:19.950 DEBUG:teuthology.task.console_log:Killing console logger for smithi050 2020-03-27T18:39:19.951 DEBUG:teuthology.task.console_log:Killing console logger for smithi155 2020-03-27T18:39:19.951 DEBUG:teuthology.task.console_log:Killing console logger for smithi164 2020-03-27T18:39:19.951 DEBUG:teuthology.exit:Finished running handlers