2020-03-27T17:53:05.918 INFO:root:teuthology version: 1.0.0-b3a6b1a 2020-03-27T17:53:05.919 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.x8zTo7.tmp -- --verbose --lock --description rados:thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/jewel.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{centos_7.6.yaml} msgr-failures/fastclose.yaml rados.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/test_rbd_api.yaml} --name sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi --block --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi/4895620 2020-03-27T17:53:05.957 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-03-27T17:53:06.016 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi/4895620 branch: revert-dec description: rados:thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/jewel.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{centos_7.6.yaml} msgr-failures/fastclose.yaml rados.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/test_rbd_api.yaml} email: null first_in_suite: false job_id: '4895620' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'librbd C and C++ api tests ' name: sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 30 - volumes: count: 4 size: 10 - volumes: size: 50 os_type: centos os_version: '7.6' overrides: admin_socket: branch: revert-dec ceph: conf: global: mon client directed command retry: 5 mon warn on msgr2 not enabled: false ms inject socket failures: 2500 ms tcp read timeout: 5 osd_pool_default_min_size: 2 osd_pool_default_size: 2 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 filestore queue throttle high multiple: 2 filestore queue throttle max multiple: 10 journal throttle high multiple: 2 journal throttle max multiple: 10 osd backoff on degraded: true osd backoff on peering: true osd debug verify cached snaps: true osd debug verify missing on start: true osd max backfills: 9 osd max markdown count: 1000 osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(REQUEST_STUCK\) - \(MON_DOWN\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request mon_bind_msgr2: false sha1: df873c1654814c23dcb68f897a012936832bfab9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: df873c1654814c23dcb68f897a012936832bfab9 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: revert-dec sha1: df873c1654814c23dcb68f897a012936832bfab9 owner: scheduled_sage@teuthology priority: 50 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - - mon.c - osd.8 - osd.9 - osd.10 - osd.11 - client.2 sha1: df873c1654814c23dcb68f897a012936832bfab9 suite: rados:thrash-old-clients suite_branch: revert-dec suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_revert-dec/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: df873c1654814c23dcb68f897a012936832bfab9 tasks: - install: branch: jewel exclude_packages: - librados3 - ceph-mgr-dashboard - ceph-mgr-diskprediction-local - ceph-mgr-rook - ceph-mgr-cephadm - cephadm - ceph-mgr - libcephfs2 - libcephfs-devel - libcephfs-dev - ceph-immutable-object-cache - ceph-base - librados-devel - python3-rados - python3-rgw - python3-rbd - python3-cephfs extra_packages: - librados2 - python-rados - python-rbd - python-cephfs - cephadm: null - exec: mon.a: - while ! ceph balancer status ; do sleep 1 ; done - ceph balancer mode crush-compat - ceph balancer on - thrashosds: chance_pgnum_grow: 3 chance_pgpnum_fix: 1 timeout: 1200 teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.2756 workload: - workunit: clients: client.2: - rbd/test_librbd.sh 2020-03-27T17:53:06.016 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_revert-dec/qa; will attempt to use it 2020-03-27T17:53:06.016 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_revert-dec/qa/tasks 2020-03-27T17:53:06.017 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-03-27T17:53:06.018 INFO:teuthology.task.internal:Checking packages... 2020-03-27T17:53:06.031 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash 'df873c1654814c23dcb68f897a012936832bfab9' 2020-03-27T17:53:06.031 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-03-27T17:53:06.031 INFO:teuthology.packaging:ref: None 2020-03-27T17:53:06.031 INFO:teuthology.packaging:tag: None 2020-03-27T17:53:06.031 INFO:teuthology.packaging:branch: revert-dec 2020-03-27T17:53:06.031 INFO:teuthology.packaging:sha1: df873c1654814c23dcb68f897a012936832bfab9 2020-03-27T17:53:06.032 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&ref=revert-dec 2020-03-27T17:53:06.251 INFO:teuthology.task.internal:Found packages for ceph version 15.2.0-377.gdf873c1 2020-03-27T17:53:06.251 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-03-27T17:53:06.261 INFO:teuthology.task.internal:no buildpackages task found 2020-03-27T17:53:06.261 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-03-27T17:53:06.311 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-03-27T17:53:06.311 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-03-27T17:53:06.355 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:53:16.367 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:53:26.381 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:53:36.393 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:53:46.417 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:53:56.436 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:54:06.475 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:54:16.494 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:54:26.511 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:54:36.521 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:54:46.562 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:54:56.579 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:55:06.605 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:55:16.624 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:55:26.642 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:55:36.670 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:55:46.685 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:55:56.701 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:56:06.724 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T17:56:16.747 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T17:56:26.762 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T17:56:36.777 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T17:56:46.794 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T17:56:56.811 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T17:57:06.843 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T17:57:16.865 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T17:57:26.884 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T17:57:36.905 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T17:57:46.926 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T17:57:56.945 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T17:58:06.963 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T17:58:16.981 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T17:58:27.000 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:58:37.029 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T17:58:47.045 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 0)... 2020-03-27T17:58:57.066 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 0)... 2020-03-27T17:59:07.082 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 0)... 2020-03-27T17:59:17.099 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 0)... 2020-03-27T17:59:27.119 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 0)... 2020-03-27T17:59:37.137 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 0)... 2020-03-27T17:59:47.158 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 0)... 2020-03-27T17:59:57.176 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 0)... 2020-03-27T18:00:07.194 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 0)... 2020-03-27T18:00:17.212 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:00:27.231 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:00:37.249 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:00:47.266 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:00:57.285 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:01:07.299 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:01:17.314 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:01:27.331 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:01:37.350 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:01:47.367 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:01:57.392 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:02:07.409 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:02:17.427 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:02:27.442 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:02:37.460 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:02:47.477 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:02:57.493 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:03:07.516 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:03:17.533 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:03:27.549 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:03:37.567 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:03:47.582 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:03:57.598 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:04:07.620 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:04:17.639 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:04:27.659 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:04:37.682 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:04:47.699 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:04:57.713 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:05:07.732 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:05:17.753 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:05:27.770 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:05:37.785 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:05:47.802 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:05:57.817 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:06:07.836 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:06:17.860 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:06:27.893 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:06:37.911 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:06:47.929 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:06:57.946 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:07:07.964 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:07:17.981 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:07:28.001 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:07:38.020 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:07:48.036 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:07:58.053 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:08:08.069 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:08:18.086 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:08:28.103 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:08:38.123 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:08:48.137 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:08:58.153 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:09:08.168 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:09:18.198 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:09:28.214 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:09:38.234 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:09:48.243 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:09:58.262 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:10:08.277 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 0)... 2020-03-27T18:10:18.294 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:10:28.308 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:10:38.331 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:10:48.352 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:10:58.369 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:11:08.385 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:11:18.402 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:11:28.417 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:11:38.441 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:11:48.462 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:11:58.477 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:12:08.497 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:12:18.514 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:12:28.537 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:12:38.553 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:12:48.579 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:12:58.594 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:13:08.610 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:13:18.630 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:13:28.646 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:13:38.655 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:13:48.677 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:13:58.697 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 4)... 2020-03-27T18:14:08.721 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:14:18.745 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:14:28.763 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:14:38.786 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:14:48.806 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:14:58.824 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:15:08.847 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:15:18.866 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:15:28.882 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:15:38.900 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:15:48.916 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:15:58.933 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:16:08.953 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 4)... 2020-03-27T18:16:18.972 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:16:28.990 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:16:39.005 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:16:49.022 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:16:59.047 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:17:09.062 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:17:19.078 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:17:29.121 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:17:39.141 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:17:49.157 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:17:59.174 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:18:09.189 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:18:19.208 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:18:29.228 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:18:39.246 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:18:49.263 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:18:59.281 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:19:09.298 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:19:19.313 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:19:29.331 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:19:39.349 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:19:49.375 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:19:59.412 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:20:09.429 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:20:19.450 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:20:29.466 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:20:39.488 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:20:49.523 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:20:59.537 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:21:09.558 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:21:19.582 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:21:29.598 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:21:39.617 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:21:49.635 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:21:59.657 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:22:09.673 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:22:19.693 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:22:29.709 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:22:39.729 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:22:49.747 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:22:59.765 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:23:09.790 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:23:19.810 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:23:29.826 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:23:39.845 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:23:49.889 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:23:59.907 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:24:09.925 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:24:19.943 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:24:29.958 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:24:39.977 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:24:49.997 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:25:00.013 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:25:10.029 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:25:20.048 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:25:30.065 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:25:40.081 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:25:50.137 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:26:00.154 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:26:10.169 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:26:20.186 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 4)... 2020-03-27T18:26:30.201 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:26:40.217 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:26:50.233 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:27:00.251 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:27:10.265 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:27:20.281 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:27:30.296 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:27:40.307 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:27:50.319 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:28:00.336 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:28:10.353 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:28:20.390 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:28:30.407 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:28:40.425 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:28:50.444 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:29:00.463 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:29:10.477 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:29:20.493 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:29:30.515 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:29:40.531 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 4)... 2020-03-27T18:29:50.545 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:30:00.562 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:30:10.578 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2020-03-27T18:30:20.594 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2020-03-27T18:30:30.612 DEBUG:teuthology.lock.ops:lock_many request: {'count': 3, 'locked_by': 'scheduled_sage@teuthology', 'description': '/home/teuthworker/archive/sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi/4895620', 'machine_type': 'smithi'} 2020-03-27T18:30:30.632 DEBUG:teuthology.lock.ops:locked ubuntu@smithi013.front.sepia.ceph.com, ubuntu@smithi175.front.sepia.ceph.com, ubuntu@smithi088.front.sepia.ceph.com 2020-03-27T18:30:30.722 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2020-03-27T18:30:30.916 INFO:teuthology.provision.fog.smithi175:Scheduling deploy of centos 7.6 2020-03-27T18:30:30.919 INFO:teuthology.provision.fog.smithi013:Scheduling deploy of centos 7.6 2020-03-27T18:30:30.921 INFO:teuthology.provision.fog.smithi088:Scheduling deploy of centos 7.6 2020-03-27T18:30:31.233 INFO:teuthology.orchestra.console:Power off smithi175 2020-03-27T18:30:31.233 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-03-27T18:30:31.266 INFO:teuthology.orchestra.console:Power off smithi088 2020-03-27T18:30:31.266 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-03-27T18:30:31.284 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:30:31.292 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:30:31.307 INFO:teuthology.orchestra.console:Power off smithi013 2020-03-27T18:30:31.308 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi013.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-03-27T18:30:31.328 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi013.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:30:31.417 INFO:teuthology.orchestra.console:Power off for smithi088 completed 2020-03-27T18:30:31.448 INFO:teuthology.orchestra.console:Power off for smithi013 completed 2020-03-27T18:30:31.518 INFO:teuthology.orchestra.console:Power on smithi088 2020-03-27T18:30:31.518 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-03-27T18:30:31.540 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:30:31.549 INFO:teuthology.orchestra.console:Power on smithi013 2020-03-27T18:30:31.549 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi013.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-03-27T18:30:31.569 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi013.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:30:35.320 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:30:35.561 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:30:35.610 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi013.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:30:39.433 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:30:39.644 INFO:teuthology.orchestra.console:Power off for smithi175 completed 2020-03-27T18:30:39.670 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:30:39.721 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi013.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:30:39.745 INFO:teuthology.orchestra.console:Power on smithi175 2020-03-27T18:30:39.745 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-03-27T18:30:39.766 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:30:39.889 INFO:teuthology.orchestra.console:Power on for smithi088 completed 2020-03-27T18:30:39.931 INFO:teuthology.orchestra.console:Power on for smithi013 completed 2020-03-27T18:30:39.990 INFO:teuthology.provision.fog.smithi088:Waiting for deploy to finish 2020-03-27T18:30:40.031 INFO:teuthology.provision.fog.smithi013:Waiting for deploy to finish 2020-03-27T18:30:43.788 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:30:47.901 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-03-27T18:30:48.111 INFO:teuthology.orchestra.console:Power on for smithi175 completed 2020-03-27T18:30:48.213 INFO:teuthology.provision.fog.smithi175:Waiting for deploy to finish 2020-03-27T18:32:56.170 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi013.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:33:05.225 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi013.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:33:14.278 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi013.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:33:19.435 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi175.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:33:23.370 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi013.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:33:26.370 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi088.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:33:28.518 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi175.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:33:32.423 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi013.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:33:35.432 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi088.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:33:37.607 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi175.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:33:41.478 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi013.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:33:44.486 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi088.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:33:46.662 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi175.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:33:53.547 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi088.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:33:55.719 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi175.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:34:02.598 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi088.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:34:04.775 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi175.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:34:11.686 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi088.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:34:13.830 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi175.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:34:20.749 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi088.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:34:29.833 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi088.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:34:38.889 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi088.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:34:47.482 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi013.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:34:47.715 INFO:teuthology.orchestra.run.smithi013:> hostname 2020-03-27T18:34:47.942 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi088.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:34:48.253 INFO:teuthology.orchestra.run.smithi013.stdout:smithi013.front.sepia.ceph.com 2020-03-27T18:34:48.253 INFO:teuthology.orchestra.run.smithi013:> grep smithi013.front.sepia.ceph.com /etc/hosts 2020-03-27T18:34:48.346 INFO:teuthology.orchestra.run.smithi013.stdout:172.21.15.13 smithi013.front.sepia.ceph.com smithi013 2020-03-27T18:34:48.347 INFO:teuthology.orchestra.run.smithi013:> sudo hostname smithi013 2020-03-27T18:34:48.495 INFO:teuthology.orchestra.run.smithi013:> sudo sed -i -e 's/smithi013.front.sepia.ceph.com/smithi013/g' /etc/hosts 2020-03-27T18:34:48.612 INFO:teuthology.orchestra.run.smithi013:> sudo sed -i -e 's/172.21.15.13/172.21.15.13/g' /etc/hosts 2020-03-27T18:34:48.729 INFO:teuthology.orchestra.run.smithi013:> sudo sed -i -e 's/smithi013.front.sepia.ceph.com/smithi013/g' /etc/hostname 2020-03-27T18:34:48.821 INFO:teuthology.orchestra.run.smithi013:> sudo hostname smithi013 2020-03-27T18:34:48.922 INFO:teuthology.provision.fog.smithi013:Deploy complete! 2020-03-27T18:34:48.926 DEBUG:teuthology.parallel:result is None 2020-03-27T18:34:51.338 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi175.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:34:51.583 INFO:teuthology.orchestra.run.smithi175:> hostname 2020-03-27T18:34:52.071 INFO:teuthology.orchestra.run.smithi175.stdout:smithi125 2020-03-27T18:34:52.072 INFO:teuthology.orchestra.run.smithi175:> grep smithi125 /etc/hosts 2020-03-27T18:34:52.129 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.125 smithi125.front.sepia.ceph.com smithi125 2020-03-27T18:34:52.129 INFO:teuthology.orchestra.run.smithi175:> sudo hostname smithi175 2020-03-27T18:34:52.240 INFO:teuthology.orchestra.run.smithi175:> sudo sed -i -e 's/smithi125/smithi175/g' /etc/hosts 2020-03-27T18:34:52.302 INFO:teuthology.orchestra.run.smithi175:> sudo sed -i -e 's/172.21.15.125/172.21.15.175/g' /etc/hosts 2020-03-27T18:34:52.358 INFO:teuthology.orchestra.run.smithi175:> sudo sed -i -e 's/smithi125/smithi175/g' /etc/hostname 2020-03-27T18:34:52.415 INFO:teuthology.orchestra.run.smithi175:> sudo hostname smithi175 2020-03-27T18:34:52.471 INFO:teuthology.provision.fog.smithi175:Deploy complete! 2020-03-27T18:34:52.472 DEBUG:teuthology.parallel:result is None 2020-03-27T18:35:25.385 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi088.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:25.603 INFO:teuthology.orchestra.run.smithi088:> hostname 2020-03-27T18:35:26.037 INFO:teuthology.orchestra.run.smithi088.stdout:smithi125 2020-03-27T18:35:26.037 INFO:teuthology.orchestra.run.smithi088:> grep smithi125 /etc/hosts 2020-03-27T18:35:26.079 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.125 smithi125.front.sepia.ceph.com smithi125 2020-03-27T18:35:26.079 INFO:teuthology.orchestra.run.smithi088:> sudo hostname smithi088 2020-03-27T18:35:26.216 INFO:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/smithi125/smithi088/g' /etc/hosts 2020-03-27T18:35:26.274 INFO:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/172.21.15.125/172.21.15.88/g' /etc/hosts 2020-03-27T18:35:26.330 INFO:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/smithi125/smithi088/g' /etc/hostname 2020-03-27T18:35:26.383 INFO:teuthology.orchestra.run.smithi088:> sudo hostname smithi088 2020-03-27T18:35:26.474 INFO:teuthology.provision.fog.smithi088:Deploy complete! 2020-03-27T18:35:26.476 DEBUG:teuthology.parallel:result is None 2020-03-27T18:35:26.703 INFO:teuthology.lock.ops:Checking smithi175.front.sepia.ceph.com 2020-03-27T18:35:26.704 INFO:teuthology.lock.ops:New key found. Updating... 2020-03-27T18:35:26.721 INFO:teuthology.lock.ops:Checking smithi088.front.sepia.ceph.com 2020-03-27T18:35:26.722 INFO:teuthology.lock.ops:New key found. Updating... 2020-03-27T18:35:26.733 INFO:teuthology.lock.ops:Checking smithi013.front.sepia.ceph.com 2020-03-27T18:35:26.733 INFO:teuthology.lock.ops:New key found. Updating... 2020-03-27T18:35:26.747 INFO:teuthology.lock.ops:Updating [smithi175.front.sepia.ceph.com]: set os type and version on server 2020-03-27T18:35:26.748 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-03-27T18:35:26.749 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi175.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:26.850 INFO:teuthology.orchestra.run.smithi175:> true 2020-03-27T18:35:27.253 INFO:teuthology.orchestra.run.smithi175:> uname -m 2020-03-27T18:35:27.337 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2020-03-27T18:35:27.337 INFO:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2020-03-27T18:35:27.408 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Linux" 2020-03-27T18:35:27.408 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="7 (Core)" 2020-03-27T18:35:27.409 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2020-03-27T18:35:27.409 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2020-03-27T18:35:27.409 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="7" 2020-03-27T18:35:27.409 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-03-27T18:35:27.409 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2020-03-27T18:35:27.409 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-03-27T18:35:27.409 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://www.centos.org/" 2020-03-27T18:35:27.410 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-03-27T18:35:27.410 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-03-27T18:35:27.410 INFO:teuthology.orchestra.run.smithi175.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-03-27T18:35:27.410 INFO:teuthology.orchestra.run.smithi175.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-03-27T18:35:27.410 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-03-27T18:35:27.410 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-03-27T18:35:27.410 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-03-27T18:35:27.411 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2020-03-27T18:35:27.436 INFO:teuthology.lock.ops:Updating [smithi088.front.sepia.ceph.com]: set os type and version on server 2020-03-27T18:35:27.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-03-27T18:35:27.436 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi088.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:27.520 INFO:teuthology.orchestra.run.smithi088:> true 2020-03-27T18:35:27.777 INFO:teuthology.orchestra.run.smithi088:> uname -m 2020-03-27T18:35:27.822 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2020-03-27T18:35:27.822 INFO:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2020-03-27T18:35:27.865 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Linux" 2020-03-27T18:35:27.866 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="7 (Core)" 2020-03-27T18:35:27.866 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos" 2020-03-27T18:35:27.866 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora" 2020-03-27T18:35:27.866 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="7" 2020-03-27T18:35:27.866 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-03-27T18:35:27.866 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2020-03-27T18:35:27.866 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-03-27T18:35:27.867 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://www.centos.org/" 2020-03-27T18:35:27.867 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-03-27T18:35:27.867 INFO:teuthology.orchestra.run.smithi088.stdout: 2020-03-27T18:35:27.867 INFO:teuthology.orchestra.run.smithi088.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-03-27T18:35:27.867 INFO:teuthology.orchestra.run.smithi088.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-03-27T18:35:27.867 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-03-27T18:35:27.867 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-03-27T18:35:27.867 INFO:teuthology.orchestra.run.smithi088.stdout: 2020-03-27T18:35:27.868 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2020-03-27T18:35:27.897 INFO:teuthology.lock.ops:Updating [smithi013.front.sepia.ceph.com]: set os type and version on server 2020-03-27T18:35:27.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-03-27T18:35:27.898 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi013.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:27.996 INFO:teuthology.orchestra.run.smithi013:> true 2020-03-27T18:35:28.417 INFO:teuthology.orchestra.run.smithi013:> uname -m 2020-03-27T18:35:28.505 INFO:teuthology.orchestra.run.smithi013.stdout:x86_64 2020-03-27T18:35:28.505 INFO:teuthology.orchestra.run.smithi013:> cat /etc/os-release 2020-03-27T18:35:28.581 INFO:teuthology.orchestra.run.smithi013.stdout:NAME="CentOS Linux" 2020-03-27T18:35:28.582 INFO:teuthology.orchestra.run.smithi013.stdout:VERSION="7 (Core)" 2020-03-27T18:35:28.582 INFO:teuthology.orchestra.run.smithi013.stdout:ID="centos" 2020-03-27T18:35:28.582 INFO:teuthology.orchestra.run.smithi013.stdout:ID_LIKE="rhel fedora" 2020-03-27T18:35:28.582 INFO:teuthology.orchestra.run.smithi013.stdout:VERSION_ID="7" 2020-03-27T18:35:28.582 INFO:teuthology.orchestra.run.smithi013.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-03-27T18:35:28.582 INFO:teuthology.orchestra.run.smithi013.stdout:ANSI_COLOR="0;31" 2020-03-27T18:35:28.583 INFO:teuthology.orchestra.run.smithi013.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-03-27T18:35:28.583 INFO:teuthology.orchestra.run.smithi013.stdout:HOME_URL="https://www.centos.org/" 2020-03-27T18:35:28.583 INFO:teuthology.orchestra.run.smithi013.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-03-27T18:35:28.583 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-03-27T18:35:28.583 INFO:teuthology.orchestra.run.smithi013.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-03-27T18:35:28.583 INFO:teuthology.orchestra.run.smithi013.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-03-27T18:35:28.583 INFO:teuthology.orchestra.run.smithi013.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-03-27T18:35:28.584 INFO:teuthology.orchestra.run.smithi013.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-03-27T18:35:28.584 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-03-27T18:35:28.584 INFO:teuthology.lock.ops:Updating smithi013.front.sepia.ceph.com on lock server 2020-03-27T18:35:28.607 INFO:teuthology.task.internal.lock_machines:3 smithi machines locked this try, 3/3 locked so far 2020-03-27T18:35:28.657 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi013.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCXQGOGj6hiv6Oxb74fZztXgQYd/IFJ+uxYIBIHjnpyljuAiNMYposXS6XRqHwWYE/J8PJWPruSxcGuKLP5cTxSo8AbxhhiUukxOa+1+WM18XUafK7ujiPluHTiwsGnIuvibdZtv4xmisJCnEvHk21aUPW1qZMWVLwK2OFijlziZ9+r2+OzQM+zm6fqnVv5jt2QA77PrvR6bW07OozfD/WgtyBZH8ar2Auy5LJtNgpMFHS4G1D59BA/EqxBo6YbjJs7D7sNT2k/HJfhXhSgPgOrBMxrsKh6nzhGocCsPL5OkCLeg9fCl5mmFEUYczcuHCWxZ19WZLY4ZowIVDMZTt/T smithi088.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDbqX9w962ZEauRYjzKGoVb2ov4czjLSphurnQZ6zzhbudhjm7sN+U3Q5T/ge76nr4w0tF1fFGSVJZqxZvHD0qIDM/Duy9NZmSSJLt1HQsmk8XTe9QgLaIHl3ZnVyCrgn4VbwldXF8k8ShEBtj00dQDRi44nbR3gBuH/pwfYgM/sHiiNq7n9KCZkQuC3lgiVXaLQb4iORqGGWzbDC+7EgaMbD9zTKaLSc7SBIDKp9gFLV1laQKKR06Qs6d6wnvNTHhcmT8PEzb0tIwG5mx9Lq/hXuA4Y2sG01fHaEiZBGakfkmRd1637+tzlGI1ZRvZc0C3/t5MelduLvAKjQogSLtJ smithi175.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDbr+wfloa5PZeqzjFDCSXyCWwzH6rU4mag6xhNsm/OMXgIjPTmLlas+LzWrLOdzA1PWyq3r6TntFBTID2QBJA2uRUtE1kNt5BD53w546xM8ON0VhSbP7O+Pht23w2z2/H/tD82q1x4Hm959Uss2+fvwhi68UJ7V2li/WrKBou01nT9UcTEBCTjZbmNnm3Mo/KrCgX4Fi7YCk58MJffGxsnKNpvfsayLy5KrgtV+VWsATHuEyXOlYx/qSaYnpCYX6JOuBrw9RqDKBpBNBx9WW3uEbs2HxguQ5/9mUWzRx9rsaZlKBHLvKqYSkSCkK24/FiWdXSPyla3elL+pVOB6EaV 2020-03-27T18:35:28.657 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-03-27T18:35:28.705 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-03-27T18:35:29.251 INFO:teuthology.task.internal:Saving configuration 2020-03-27T18:35:29.367 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-03-27T18:35:29.406 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-03-27T18:35:29.427 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2020-03-27 18:30:30.618792', u'locked_by': u'scheduled_sage@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi175.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDCaa/GSxpozqpIR+oTqlaU7912p9MK5xgFSsy5T3u/ge2xQc6qVy3F8iSI2Hs7wfeVYzL2KN65H4sEvAXE8q8I=', u'description': u'/home/teuthworker/archive/sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi/4895620'} 2020-03-27T18:35:29.442 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2020-03-27 18:30:30.618770', u'locked_by': u'scheduled_sage@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi088.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOHXxUezRJ1N9eTB4ufi0zhkOEesZP9beNoLJsdYuC6oWGJFD2idwhXTp2mnKSOU7ztuiScfH9uvjYu5GRt9304=', u'description': u'/home/teuthworker/archive/sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi/4895620'} 2020-03-27T18:35:29.456 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2020-03-27 18:30:30.618808', u'locked_by': u'scheduled_sage@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi013.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIDceKdFU+EXnSQhdCYVN315gHJpL7KTggPklQF3lVzRjW18OI49NG2Gti7LEDqUSUHfZU7EIS5KmaE5jd9BaOc=', u'description': u'/home/teuthworker/archive/sage-2020-03-27_17:51:50-rados:thrash-old-clients-revert-dec-distro-basic-smithi/4895620'} 2020-03-27T18:35:29.457 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-03-27T18:35:29.786 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2020-03-27T18:35:29.786 INFO:teuthology.task.internal:roles: ubuntu@smithi088.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2020-03-27T18:35:29.786 INFO:teuthology.task.internal:roles: ubuntu@smithi013.front.sepia.ceph.com - ['mon.c', 'osd.8', 'osd.9', 'osd.10', 'osd.11', 'client.2'] 2020-03-27T18:35:29.786 INFO:teuthology.run_tasks:Running task console_log... 2020-03-27T18:35:29.969 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2020-03-27T18:35:29.969 INFO:teuthology.run_tasks:Running task internal.connect... 2020-03-27T18:35:29.983 INFO:teuthology.task.internal:Opening connections... 2020-03-27T18:35:29.983 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi013.front.sepia.ceph.com 2020-03-27T18:35:29.985 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi013.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:30.090 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2020-03-27T18:35:30.091 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi175.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:30.194 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi088.front.sepia.ceph.com 2020-03-27T18:35:30.195 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi088.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:35:30.291 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-03-27T18:35:30.298 INFO:teuthology.orchestra.run.smithi013:> uname -m 2020-03-27T18:35:30.432 INFO:teuthology.orchestra.run.smithi013.stdout:x86_64 2020-03-27T18:35:30.433 INFO:teuthology.orchestra.run.smithi013:> cat /etc/os-release 2020-03-27T18:35:30.504 INFO:teuthology.orchestra.run.smithi013.stdout:NAME="CentOS Linux" 2020-03-27T18:35:30.504 INFO:teuthology.orchestra.run.smithi013.stdout:VERSION="7 (Core)" 2020-03-27T18:35:30.505 INFO:teuthology.orchestra.run.smithi013.stdout:ID="centos" 2020-03-27T18:35:30.505 INFO:teuthology.orchestra.run.smithi013.stdout:ID_LIKE="rhel fedora" 2020-03-27T18:35:30.505 INFO:teuthology.orchestra.run.smithi013.stdout:VERSION_ID="7" 2020-03-27T18:35:30.505 INFO:teuthology.orchestra.run.smithi013.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-03-27T18:35:30.505 INFO:teuthology.orchestra.run.smithi013.stdout:ANSI_COLOR="0;31" 2020-03-27T18:35:30.505 INFO:teuthology.orchestra.run.smithi013.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-03-27T18:35:30.505 INFO:teuthology.orchestra.run.smithi013.stdout:HOME_URL="https://www.centos.org/" 2020-03-27T18:35:30.506 INFO:teuthology.orchestra.run.smithi013.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-03-27T18:35:30.506 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-03-27T18:35:30.506 INFO:teuthology.orchestra.run.smithi013.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-03-27T18:35:30.506 INFO:teuthology.orchestra.run.smithi013.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-03-27T18:35:30.506 INFO:teuthology.orchestra.run.smithi013.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-03-27T18:35:30.506 INFO:teuthology.orchestra.run.smithi013.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-03-27T18:35:30.506 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-03-27T18:35:30.507 INFO:teuthology.lock.ops:Updating smithi013.front.sepia.ceph.com on lock server 2020-03-27T18:35:30.519 INFO:teuthology.orchestra.run.smithi175:> uname -m 2020-03-27T18:35:30.627 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2020-03-27T18:35:30.627 INFO:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2020-03-27T18:35:30.698 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Linux" 2020-03-27T18:35:30.698 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="7 (Core)" 2020-03-27T18:35:30.698 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2020-03-27T18:35:30.699 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2020-03-27T18:35:30.700 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="7" 2020-03-27T18:35:30.700 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-03-27T18:35:30.700 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2020-03-27T18:35:30.701 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-03-27T18:35:30.701 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://www.centos.org/" 2020-03-27T18:35:30.701 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-03-27T18:35:30.701 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-03-27T18:35:30.701 INFO:teuthology.orchestra.run.smithi175.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-03-27T18:35:30.702 INFO:teuthology.orchestra.run.smithi175.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-03-27T18:35:30.702 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-03-27T18:35:30.702 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-03-27T18:35:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-03-27T18:35:30.704 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2020-03-27T18:35:30.721 INFO:teuthology.orchestra.run.smithi088:> uname -m 2020-03-27T18:35:30.827 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2020-03-27T18:35:30.827 INFO:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2020-03-27T18:35:30.887 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Linux" 2020-03-27T18:35:30.887 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="7 (Core)" 2020-03-27T18:35:30.887 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos" 2020-03-27T18:35:30.887 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora" 2020-03-27T18:35:30.887 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="7" 2020-03-27T18:35:30.888 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2020-03-27T18:35:30.888 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2020-03-27T18:35:30.888 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2020-03-27T18:35:30.888 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://www.centos.org/" 2020-03-27T18:35:30.888 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-03-27T18:35:30.888 INFO:teuthology.orchestra.run.smithi088.stdout: 2020-03-27T18:35:30.888 INFO:teuthology.orchestra.run.smithi088.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2020-03-27T18:35:30.889 INFO:teuthology.orchestra.run.smithi088.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2020-03-27T18:35:30.889 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-03-27T18:35:30.889 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2020-03-27T18:35:30.889 INFO:teuthology.orchestra.run.smithi088.stdout: 2020-03-27T18:35:30.889 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2020-03-27T18:35:30.902 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-03-27T18:35:30.933 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-03-27T18:35:30.942 INFO:teuthology.task.internal:Checking for old test directory... 2020-03-27T18:35:30.942 INFO:teuthology.orchestra.run.smithi013:> test '!' -e /home/ubuntu/cephtest 2020-03-27T18:35:30.946 INFO:teuthology.orchestra.run.smithi088:> test '!' -e /home/ubuntu/cephtest 2020-03-27T18:35:30.952 INFO:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2020-03-27T18:35:31.039 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-03-27T18:35:31.049 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-03-27T18:35:31.050 INFO:teuthology.orchestra.run.smithi013:> test -z $(ls -A /var/lib/ceph) 2020-03-27T18:35:31.056 INFO:teuthology.orchestra.run.smithi088:> test -z $(ls -A /var/lib/ceph) 2020-03-27T18:35:31.059 INFO:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2020-03-27T18:35:31.152 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-03-27T18:35:31.206 INFO:teuthology.run_tasks:Running task kernel... 2020-03-27T18:35:31.236 DEBUG:teuthology.task.kernel:normalized config {'osd.9': {'sha1': 'distro', 'kdb': True}, 'osd.8': {'sha1': 'distro', 'kdb': True}, 'mon.c': {'sha1': 'distro', 'kdb': True}, 'client.0': {'sha1': 'distro', 'kdb': True}, 'osd.1': {'sha1': 'distro', 'kdb': True}, 'client.2': {'sha1': 'distro', 'kdb': True}, 'osd.3': {'sha1': 'distro', 'kdb': True}, 'osd.2': {'sha1': 'distro', 'kdb': True}, 'osd.5': {'sha1': 'distro', 'kdb': True}, 'osd.4': {'sha1': 'distro', 'kdb': True}, 'osd.7': {'sha1': 'distro', 'kdb': True}, 'osd.6': {'sha1': 'distro', 'kdb': True}, 'osd.11': {'sha1': 'distro', 'kdb': True}, 'osd.10': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}, 'mgr.y': {'sha1': 'distro', 'kdb': True}, 'mgr.x': {'sha1': 'distro', 'kdb': True}, 'osd.0': {'sha1': 'distro', 'kdb': True}, 'mon.a': {'sha1': 'distro', 'kdb': True}, 'client.1': {'sha1': 'distro', 'kdb': True}} 2020-03-27T18:35:31.237 INFO:teuthology.task.kernel:config {'mon.c': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}, 'mon.a': {'sha1': 'distro', 'kdb': True}}, timeout 300 2020-03-27T18:35:31.237 INFO:teuthology.orchestra.run.smithi013:> uname -r 2020-03-27T18:35:31.325 INFO:teuthology.orchestra.run.smithi013.stdout:3.10.0-1062.12.1.el7.x86_64 2020-03-27T18:35:31.326 INFO:teuthology.task.kernel:Running kernel on smithi013: 3.10.0-1062.12.1.el7.x86_64 2020-03-27T18:35:31.326 INFO:teuthology.orchestra.run.smithi013:> sudo yum install -y kernel 2020-03-27T18:35:32.259 INFO:teuthology.orchestra.run.smithi013.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-03-27T18:35:32.377 INFO:teuthology.orchestra.run.smithi013.stdout:Determining fastest mirrors 2020-03-27T18:35:35.881 INFO:teuthology.orchestra.run.smithi013.stdout: * base: mirror.net.cen.ct.gov 2020-03-27T18:35:35.891 INFO:teuthology.orchestra.run.smithi013.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-03-27T18:35:35.891 INFO:teuthology.orchestra.run.smithi013.stdout: * extras: packages.oit.ncsu.edu 2020-03-27T18:35:35.891 INFO:teuthology.orchestra.run.smithi013.stdout: * updates: www.gtlib.gatech.edu 2020-03-27T18:35:43.018 INFO:teuthology.orchestra.run.smithi013.stdout:Resolving Dependencies 2020-03-27T18:35:43.018 INFO:teuthology.orchestra.run.smithi013.stdout:--> Running transaction check 2020-03-27T18:35:43.018 INFO:teuthology.orchestra.run.smithi013.stdout:---> Package kernel.x86_64 0:3.10.0-1062.18.1.el7 will be installed 2020-03-27T18:35:44.272 INFO:teuthology.orchestra.run.smithi013.stdout:--> Finished Dependency Resolution 2020-03-27T18:35:45.032 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-03-27T18:35:45.032 INFO:teuthology.orchestra.run.smithi013.stdout:Dependencies Resolved 2020-03-27T18:35:45.033 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-03-27T18:35:45.034 INFO:teuthology.orchestra.run.smithi013.stdout:================================================================================ 2020-03-27T18:35:45.034 INFO:teuthology.orchestra.run.smithi013.stdout: Package Arch Version Repository Size 2020-03-27T18:35:45.034 INFO:teuthology.orchestra.run.smithi013.stdout:================================================================================ 2020-03-27T18:35:45.034 INFO:teuthology.orchestra.run.smithi013.stdout:Installing: 2020-03-27T18:35:45.035 INFO:teuthology.orchestra.run.smithi013.stdout: kernel x86_64 3.10.0-1062.18.1.el7 updates 50 M 2020-03-27T18:35:45.035 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-03-27T18:35:45.035 INFO:teuthology.orchestra.run.smithi013.stdout:Transaction Summary 2020-03-27T18:35:45.035 INFO:teuthology.orchestra.run.smithi013.stdout:================================================================================ 2020-03-27T18:35:45.036 INFO:teuthology.orchestra.run.smithi013.stdout:Install 1 Package 2020-03-27T18:35:45.036 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-03-27T18:35:45.036 INFO:teuthology.orchestra.run.smithi013.stdout:Total download size: 50 M 2020-03-27T18:35:45.036 INFO:teuthology.orchestra.run.smithi013.stdout:Installed size: 64 M 2020-03-27T18:35:45.036 INFO:teuthology.orchestra.run.smithi013.stdout:Downloading packages: 2020-03-27T18:35:45.050 INFO:teuthology.orchestra.run.smithi013.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2020-03-27T18:35:47.599 INFO:teuthology.orchestra.run.smithi013.stdout:Running transaction check 2020-03-27T18:35:47.798 INFO:teuthology.orchestra.run.smithi013.stdout:Running transaction test 2020-03-27T18:35:48.618 INFO:teuthology.orchestra.run.smithi013.stdout:Transaction test succeeded 2020-03-27T18:35:48.618 INFO:teuthology.orchestra.run.smithi013.stdout:Running transaction 2020-03-27T18:36:25.760 INFO:teuthology.orchestra.run.smithi013.stdout: Installing : kernel-3.10.0-1062.18.1.el7.x86_64 1/1 2020-03-27T18:36:26.364 INFO:teuthology.orchestra.run.smithi013.stdout: Verifying : kernel-3.10.0-1062.18.1.el7.x86_64 1/1 2020-03-27T18:36:26.364 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-03-27T18:36:26.364 INFO:teuthology.orchestra.run.smithi013.stdout:Installed: 2020-03-27T18:36:26.365 INFO:teuthology.orchestra.run.smithi013.stdout: kernel.x86_64 0:3.10.0-1062.18.1.el7 2020-03-27T18:36:26.365 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-03-27T18:36:26.365 INFO:teuthology.orchestra.run.smithi013.stdout:Complete! 2020-03-27T18:36:26.413 INFO:teuthology.orchestra.run.smithi013:> rpm -q kernel --last | head -n 1 2020-03-27T18:36:26.826 INFO:teuthology.orchestra.run.smithi013.stdout:kernel-3.10.0-1062.18.1.el7.x86_64 Fri 27 Mar 2020 06:35:51 PM UTC 2020-03-27T18:36:26.829 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:36:26.829 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.12.1.el7.x86_64 Expected: 3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:36:26.830 INFO:teuthology.orchestra.run.smithi088:> uname -r 2020-03-27T18:36:26.956 INFO:teuthology.orchestra.run.smithi088.stdout:3.10.0-1062.12.1.el7.x86_64 2020-03-27T18:36:26.956 INFO:teuthology.task.kernel:Running kernel on smithi088: 3.10.0-1062.12.1.el7.x86_64 2020-03-27T18:36:26.956 INFO:teuthology.orchestra.run.smithi088:> sudo yum install -y kernel 2020-03-27T18:36:28.145 INFO:teuthology.orchestra.run.smithi088.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-03-27T18:36:28.244 INFO:teuthology.orchestra.run.smithi088.stdout:Determining fastest mirrors 2020-03-27T18:36:32.394 INFO:teuthology.orchestra.run.smithi088.stdout: * base: mirror.net.cen.ct.gov 2020-03-27T18:36:32.400 INFO:teuthology.orchestra.run.smithi088.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-03-27T18:36:32.400 INFO:teuthology.orchestra.run.smithi088.stdout: * extras: packages.oit.ncsu.edu 2020-03-27T18:36:32.401 INFO:teuthology.orchestra.run.smithi088.stdout: * updates: www.gtlib.gatech.edu 2020-03-27T18:36:48.257 INFO:teuthology.orchestra.run.smithi088.stdout:Resolving Dependencies 2020-03-27T18:36:48.258 INFO:teuthology.orchestra.run.smithi088.stdout:--> Running transaction check 2020-03-27T18:36:48.258 INFO:teuthology.orchestra.run.smithi088.stdout:---> Package kernel.x86_64 0:3.10.0-1062.18.1.el7 will be installed 2020-03-27T18:36:49.513 INFO:teuthology.orchestra.run.smithi088.stdout:--> Finished Dependency Resolution 2020-03-27T18:36:50.331 INFO:teuthology.orchestra.run.smithi088.stdout: 2020-03-27T18:36:50.331 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies Resolved 2020-03-27T18:36:50.342 INFO:teuthology.orchestra.run.smithi088.stdout: 2020-03-27T18:36:50.342 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2020-03-27T18:36:50.342 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2020-03-27T18:36:50.342 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2020-03-27T18:36:50.343 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2020-03-27T18:36:50.343 INFO:teuthology.orchestra.run.smithi088.stdout: kernel x86_64 3.10.0-1062.18.1.el7 updates 50 M 2020-03-27T18:36:50.343 INFO:teuthology.orchestra.run.smithi088.stdout: 2020-03-27T18:36:50.343 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2020-03-27T18:36:50.343 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2020-03-27T18:36:50.343 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2020-03-27T18:36:50.344 INFO:teuthology.orchestra.run.smithi088.stdout: 2020-03-27T18:36:50.344 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 50 M 2020-03-27T18:36:50.344 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 64 M 2020-03-27T18:36:50.344 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading packages: 2020-03-27T18:36:50.357 INFO:teuthology.orchestra.run.smithi088.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2020-03-27T18:36:51.836 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2020-03-27T18:36:52.026 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2020-03-27T18:36:52.942 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded 2020-03-27T18:36:52.943 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2020-03-27T18:37:28.793 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : kernel-3.10.0-1062.18.1.el7.x86_64 1/1 2020-03-27T18:37:29.336 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : kernel-3.10.0-1062.18.1.el7.x86_64 1/1 2020-03-27T18:37:29.336 INFO:teuthology.orchestra.run.smithi088.stdout: 2020-03-27T18:37:29.336 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2020-03-27T18:37:29.337 INFO:teuthology.orchestra.run.smithi088.stdout: kernel.x86_64 0:3.10.0-1062.18.1.el7 2020-03-27T18:37:29.337 INFO:teuthology.orchestra.run.smithi088.stdout: 2020-03-27T18:37:29.337 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2020-03-27T18:37:29.388 INFO:teuthology.orchestra.run.smithi088:> rpm -q kernel --last | head -n 1 2020-03-27T18:37:29.770 INFO:teuthology.orchestra.run.smithi088.stdout:kernel-3.10.0-1062.18.1.el7.x86_64 Fri 27 Mar 2020 06:36:55 PM UTC 2020-03-27T18:37:29.773 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:37:29.773 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.12.1.el7.x86_64 Expected: 3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:37:29.773 INFO:teuthology.orchestra.run.smithi175:> uname -r 2020-03-27T18:37:29.887 INFO:teuthology.orchestra.run.smithi175.stdout:3.10.0-1062.12.1.el7.x86_64 2020-03-27T18:37:29.888 INFO:teuthology.task.kernel:Running kernel on smithi175: 3.10.0-1062.12.1.el7.x86_64 2020-03-27T18:37:29.888 INFO:teuthology.orchestra.run.smithi175:> sudo yum install -y kernel 2020-03-27T18:37:30.789 INFO:teuthology.orchestra.run.smithi175.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2020-03-27T18:37:30.909 INFO:teuthology.orchestra.run.smithi175.stdout:Determining fastest mirrors 2020-03-27T18:37:34.221 INFO:teuthology.orchestra.run.smithi175.stdout: * base: repos-va.psychz.net 2020-03-27T18:37:34.232 INFO:teuthology.orchestra.run.smithi175.stdout: * epel: download-cc-rdu01.fedoraproject.org 2020-03-27T18:37:34.233 INFO:teuthology.orchestra.run.smithi175.stdout: * extras: packages.oit.ncsu.edu 2020-03-27T18:37:34.233 INFO:teuthology.orchestra.run.smithi175.stdout: * updates: www.gtlib.gatech.edu 2020-03-27T18:37:40.214 INFO:teuthology.orchestra.run.smithi175.stdout:Resolving Dependencies 2020-03-27T18:37:40.214 INFO:teuthology.orchestra.run.smithi175.stdout:--> Running transaction check 2020-03-27T18:37:40.215 INFO:teuthology.orchestra.run.smithi175.stdout:---> Package kernel.x86_64 0:3.10.0-1062.18.1.el7 will be installed 2020-03-27T18:37:41.491 INFO:teuthology.orchestra.run.smithi175.stdout:--> Finished Dependency Resolution 2020-03-27T18:37:42.355 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-03-27T18:37:42.355 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies Resolved 2020-03-27T18:37:42.357 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-03-27T18:37:42.357 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2020-03-27T18:37:42.358 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2020-03-27T18:37:42.358 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2020-03-27T18:37:42.358 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2020-03-27T18:37:42.358 INFO:teuthology.orchestra.run.smithi175.stdout: kernel x86_64 3.10.0-1062.18.1.el7 updates 50 M 2020-03-27T18:37:42.358 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-03-27T18:37:42.358 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2020-03-27T18:37:42.358 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2020-03-27T18:37:42.359 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2020-03-27T18:37:42.359 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-03-27T18:37:42.359 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 50 M 2020-03-27T18:37:42.359 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 64 M 2020-03-27T18:37:42.359 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading packages: 2020-03-27T18:37:42.367 INFO:teuthology.orchestra.run.smithi175.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2020-03-27T18:37:44.867 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2020-03-27T18:37:45.060 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2020-03-27T18:37:46.009 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded 2020-03-27T18:37:46.009 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2020-03-27T18:38:21.160 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-3.10.0-1062.18.1.el7.x86_64 1/1 2020-03-27T18:38:21.727 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-3.10.0-1062.18.1.el7.x86_64 1/1 2020-03-27T18:38:21.727 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-03-27T18:38:21.727 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2020-03-27T18:38:21.728 INFO:teuthology.orchestra.run.smithi175.stdout: kernel.x86_64 0:3.10.0-1062.18.1.el7 2020-03-27T18:38:21.728 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-03-27T18:38:21.728 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2020-03-27T18:38:21.802 INFO:teuthology.orchestra.run.smithi175:> rpm -q kernel --last | head -n 1 2020-03-27T18:38:22.150 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-3.10.0-1062.18.1.el7.x86_64 Fri 27 Mar 2020 06:37:48 PM UTC 2020-03-27T18:38:22.152 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:38:22.152 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.12.1.el7.x86_64 Expected: 3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:38:22.152 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2020-03-27T18:38:22.152 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-03-27T18:38:22.152 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-03-27T18:38:22.153 DEBUG:teuthology.task.kernel:src is distro, skipping download 2020-03-27T18:38:22.153 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2020-03-27T18:38:22.153 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi175.front.sepia.ceph.com, path=None, version=distro) 2020-03-27T18:38:22.153 INFO:teuthology.orchestra.run.smithi175:> rpm -q kernel --last | head -n 1 2020-03-27T18:38:22.464 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-3.10.0-1062.18.1.el7.x86_64 Fri 27 Mar 2020 06:37:48 PM UTC 2020-03-27T18:38:22.467 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:38:22.467 INFO:teuthology.orchestra.run.smithi175:> sudo rpm -qi grub2-tools 2020-03-27T18:38:22.604 INFO:teuthology.orchestra.run.smithi175.stdout:Name : grub2-tools 2020-03-27T18:38:22.605 INFO:teuthology.orchestra.run.smithi175.stdout:Epoch : 1 2020-03-27T18:38:22.605 INFO:teuthology.orchestra.run.smithi175.stdout:Version : 2.02 2020-03-27T18:38:22.605 INFO:teuthology.orchestra.run.smithi175.stdout:Release : 0.76.el7.centos 2020-03-27T18:38:22.605 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: x86_64 2020-03-27T18:38:22.605 INFO:teuthology.orchestra.run.smithi175.stdout:Install Date: Tue 11 Feb 2020 03:34:40 PM UTC 2020-03-27T18:38:22.606 INFO:teuthology.orchestra.run.smithi175.stdout:Group : System Environment/Base 2020-03-27T18:38:22.606 INFO:teuthology.orchestra.run.smithi175.stdout:Size : 10080361 2020-03-27T18:38:22.606 INFO:teuthology.orchestra.run.smithi175.stdout:License : GPLv3+ 2020-03-27T18:38:22.606 INFO:teuthology.orchestra.run.smithi175.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:11 PM UTC, Key ID 24c6a8a7f4a80eb5 2020-03-27T18:38:22.606 INFO:teuthology.orchestra.run.smithi175.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2020-03-27T18:38:22.606 INFO:teuthology.orchestra.run.smithi175.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2020-03-27T18:38:22.606 INFO:teuthology.orchestra.run.smithi175.stdout:Build Host : kbuilder.bsys.centos.org 2020-03-27T18:38:22.607 INFO:teuthology.orchestra.run.smithi175.stdout:Relocations : (not relocatable) 2020-03-27T18:38:22.607 INFO:teuthology.orchestra.run.smithi175.stdout:Packager : CentOS BuildSystem 2020-03-27T18:38:22.607 INFO:teuthology.orchestra.run.smithi175.stdout:Vendor : CentOS 2020-03-27T18:38:22.607 INFO:teuthology.orchestra.run.smithi175.stdout:URL : http://www.gnu.org/software/grub/ 2020-03-27T18:38:22.607 INFO:teuthology.orchestra.run.smithi175.stdout:Summary : Support tools for GRUB. 2020-03-27T18:38:22.608 INFO:teuthology.orchestra.run.smithi175.stdout:Description : 2020-03-27T18:38:22.608 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-03-27T18:38:22.608 INFO:teuthology.orchestra.run.smithi175.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2020-03-27T18:38:22.608 INFO:teuthology.orchestra.run.smithi175.stdout:customizable bootloader with modular architecture. It supports a rich 2020-03-27T18:38:22.608 INFO:teuthology.orchestra.run.smithi175.stdout:variety of kernel formats, file systems, computer architectures and 2020-03-27T18:38:22.608 INFO:teuthology.orchestra.run.smithi175.stdout:hardware devices. 2020-03-27T18:38:22.609 INFO:teuthology.orchestra.run.smithi175.stdout: 2020-03-27T18:38:22.609 INFO:teuthology.orchestra.run.smithi175.stdout:This subpackage provides tools for support of all platforms. 2020-03-27T18:38:22.610 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2020-03-27T18:38:22.610 INFO:teuthology.task.kernel:Updating grub on smithi175 to boot 3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:38:22.610 INFO:teuthology.orchestra.run.smithi175:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2020-03-27T18:38:23.079 INFO:teuthology.orchestra.run.smithi175.stderr:Generating grub configuration file ... 2020-03-27T18:38:23.212 INFO:teuthology.orchestra.run.smithi175.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:38:23.228 INFO:teuthology.orchestra.run.smithi175.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.18.1.el7.x86_64.img 2020-03-27T18:38:23.492 INFO:teuthology.orchestra.run.smithi175.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.12.1.el7.x86_64 2020-03-27T18:38:23.519 INFO:teuthology.orchestra.run.smithi175.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.12.1.el7.x86_64.img 2020-03-27T18:38:23.548 INFO:teuthology.orchestra.run.smithi175.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2020-03-27T18:38:23.566 INFO:teuthology.orchestra.run.smithi175.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2020-03-27T18:38:23.593 INFO:teuthology.orchestra.run.smithi175.stderr:Found linux image: /boot/vmlinuz-0-rescue-11867a3080154962a87601c586b1198d 2020-03-27T18:38:23.602 INFO:teuthology.orchestra.run.smithi175.stderr:Found initrd image: /boot/initramfs-0-rescue-11867a3080154962a87601c586b1198d.img 2020-03-27T18:38:24.840 INFO:teuthology.orchestra.run.smithi175.stderr:done 2020-03-27T18:38:24.842 INFO:teuthology.orchestra.run.smithi175:> mktemp 2020-03-27T18:38:24.923 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.9UIfGOyeHc 2020-03-27T18:38:24.924 INFO:teuthology.orchestra.run.smithi175:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9UIfGOyeHc 2020-03-27T18:38:25.036 INFO:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.9UIfGOyeHc 2020-03-27T18:38:25.257 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.9UIfGOyeHc is 6KB 2020-03-27T18:38:25.358 INFO:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.9UIfGOyeHc 2020-03-27T18:38:25.435 INFO:teuthology.orchestra.run.smithi175:> sudo grub2-set-default 0 2020-03-27T18:38:25.637 INFO:teuthology.orchestra.run.smithi175:> sudo shutdown -r now 2020-03-27T18:38:25.641 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2020-03-27T18:38:25.641 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi013.front.sepia.ceph.com, path=None, version=distro) 2020-03-27T18:38:25.642 INFO:teuthology.orchestra.run.smithi013:> rpm -q kernel --last | head -n 1 2020-03-27T18:38:26.006 INFO:teuthology.orchestra.run.smithi013.stdout:kernel-3.10.0-1062.18.1.el7.x86_64 Fri 27 Mar 2020 06:35:51 PM UTC 2020-03-27T18:38:26.008 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:38:26.008 INFO:teuthology.orchestra.run.smithi013:> sudo rpm -qi grub2-tools 2020-03-27T18:38:26.156 INFO:teuthology.orchestra.run.smithi013.stdout:Name : grub2-tools 2020-03-27T18:38:26.156 INFO:teuthology.orchestra.run.smithi013.stdout:Epoch : 1 2020-03-27T18:38:26.157 INFO:teuthology.orchestra.run.smithi013.stdout:Version : 2.02 2020-03-27T18:38:26.157 INFO:teuthology.orchestra.run.smithi013.stdout:Release : 0.76.el7.centos 2020-03-27T18:38:26.157 INFO:teuthology.orchestra.run.smithi013.stdout:Architecture: x86_64 2020-03-27T18:38:26.157 INFO:teuthology.orchestra.run.smithi013.stdout:Install Date: Tue 11 Feb 2020 03:34:40 PM UTC 2020-03-27T18:38:26.157 INFO:teuthology.orchestra.run.smithi013.stdout:Group : System Environment/Base 2020-03-27T18:38:26.157 INFO:teuthology.orchestra.run.smithi013.stdout:Size : 10080361 2020-03-27T18:38:26.157 INFO:teuthology.orchestra.run.smithi013.stdout:License : GPLv3+ 2020-03-27T18:38:26.158 INFO:teuthology.orchestra.run.smithi013.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:11 PM UTC, Key ID 24c6a8a7f4a80eb5 2020-03-27T18:38:26.158 INFO:teuthology.orchestra.run.smithi013.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2020-03-27T18:38:26.158 INFO:teuthology.orchestra.run.smithi013.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2020-03-27T18:38:26.158 INFO:teuthology.orchestra.run.smithi013.stdout:Build Host : kbuilder.bsys.centos.org 2020-03-27T18:38:26.158 INFO:teuthology.orchestra.run.smithi013.stdout:Relocations : (not relocatable) 2020-03-27T18:38:26.158 INFO:teuthology.orchestra.run.smithi013.stdout:Packager : CentOS BuildSystem 2020-03-27T18:38:26.158 INFO:teuthology.orchestra.run.smithi013.stdout:Vendor : CentOS 2020-03-27T18:38:26.158 INFO:teuthology.orchestra.run.smithi013.stdout:URL : http://www.gnu.org/software/grub/ 2020-03-27T18:38:26.159 INFO:teuthology.orchestra.run.smithi013.stdout:Summary : Support tools for GRUB. 2020-03-27T18:38:26.159 INFO:teuthology.orchestra.run.smithi013.stdout:Description : 2020-03-27T18:38:26.159 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-03-27T18:38:26.159 INFO:teuthology.orchestra.run.smithi013.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2020-03-27T18:38:26.159 INFO:teuthology.orchestra.run.smithi013.stdout:customizable bootloader with modular architecture. It supports a rich 2020-03-27T18:38:26.159 INFO:teuthology.orchestra.run.smithi013.stdout:variety of kernel formats, file systems, computer architectures and 2020-03-27T18:38:26.159 INFO:teuthology.orchestra.run.smithi013.stdout:hardware devices. 2020-03-27T18:38:26.160 INFO:teuthology.orchestra.run.smithi013.stdout: 2020-03-27T18:38:26.160 INFO:teuthology.orchestra.run.smithi013.stdout:This subpackage provides tools for support of all platforms. 2020-03-27T18:38:26.161 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2020-03-27T18:38:26.161 INFO:teuthology.task.kernel:Updating grub on smithi013 to boot 3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:38:26.161 INFO:teuthology.orchestra.run.smithi013:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2020-03-27T18:38:26.677 INFO:teuthology.orchestra.run.smithi013.stderr:Generating grub configuration file ... 2020-03-27T18:38:26.807 INFO:teuthology.orchestra.run.smithi013.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:38:26.823 INFO:teuthology.orchestra.run.smithi013.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.18.1.el7.x86_64.img 2020-03-27T18:38:27.092 INFO:teuthology.orchestra.run.smithi013.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.12.1.el7.x86_64 2020-03-27T18:38:27.111 INFO:teuthology.orchestra.run.smithi013.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.12.1.el7.x86_64.img 2020-03-27T18:38:27.146 INFO:teuthology.orchestra.run.smithi013.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2020-03-27T18:38:27.161 INFO:teuthology.orchestra.run.smithi013.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2020-03-27T18:38:27.190 INFO:teuthology.orchestra.run.smithi013.stderr:Found linux image: /boot/vmlinuz-0-rescue-11867a3080154962a87601c586b1198d 2020-03-27T18:38:27.200 INFO:teuthology.orchestra.run.smithi013.stderr:Found initrd image: /boot/initramfs-0-rescue-11867a3080154962a87601c586b1198d.img 2020-03-27T18:38:28.066 INFO:teuthology.orchestra.run.smithi013.stderr:done 2020-03-27T18:38:28.068 INFO:teuthology.orchestra.run.smithi013:> mktemp 2020-03-27T18:38:28.137 INFO:teuthology.orchestra.run.smithi013.stdout:/tmp/tmp.hUU4M6RCF0 2020-03-27T18:38:28.137 INFO:teuthology.orchestra.run.smithi013:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.hUU4M6RCF0 2020-03-27T18:38:28.216 INFO:teuthology.orchestra.run.smithi013:> sudo chmod 0666 /tmp/tmp.hUU4M6RCF0 2020-03-27T18:38:28.362 DEBUG:teuthology.orchestra.remote:smithi013:/tmp/tmp.hUU4M6RCF0 is 6KB 2020-03-27T18:38:28.421 INFO:teuthology.orchestra.run.smithi013:> rm -fr /tmp/tmp.hUU4M6RCF0 2020-03-27T18:38:28.477 INFO:teuthology.orchestra.run.smithi013:> sudo grub2-set-default 0 2020-03-27T18:38:28.627 INFO:teuthology.orchestra.run.smithi013:> sudo shutdown -r now 2020-03-27T18:38:28.630 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2020-03-27T18:38:28.631 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi088.front.sepia.ceph.com, path=None, version=distro) 2020-03-27T18:38:28.631 INFO:teuthology.orchestra.run.smithi088:> rpm -q kernel --last | head -n 1 2020-03-27T18:38:28.988 INFO:teuthology.orchestra.run.smithi088.stdout:kernel-3.10.0-1062.18.1.el7.x86_64 Fri 27 Mar 2020 06:36:55 PM UTC 2020-03-27T18:38:28.990 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:38:28.991 INFO:teuthology.orchestra.run.smithi088:> sudo rpm -qi grub2-tools 2020-03-27T18:38:29.149 INFO:teuthology.orchestra.run.smithi088.stdout:Name : grub2-tools 2020-03-27T18:38:29.150 INFO:teuthology.orchestra.run.smithi088.stdout:Epoch : 1 2020-03-27T18:38:29.150 INFO:teuthology.orchestra.run.smithi088.stdout:Version : 2.02 2020-03-27T18:38:29.150 INFO:teuthology.orchestra.run.smithi088.stdout:Release : 0.76.el7.centos 2020-03-27T18:38:29.150 INFO:teuthology.orchestra.run.smithi088.stdout:Architecture: x86_64 2020-03-27T18:38:29.150 INFO:teuthology.orchestra.run.smithi088.stdout:Install Date: Tue 11 Feb 2020 03:34:40 PM UTC 2020-03-27T18:38:29.150 INFO:teuthology.orchestra.run.smithi088.stdout:Group : System Environment/Base 2020-03-27T18:38:29.151 INFO:teuthology.orchestra.run.smithi088.stdout:Size : 10080361 2020-03-27T18:38:29.151 INFO:teuthology.orchestra.run.smithi088.stdout:License : GPLv3+ 2020-03-27T18:38:29.151 INFO:teuthology.orchestra.run.smithi088.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:11 PM UTC, Key ID 24c6a8a7f4a80eb5 2020-03-27T18:38:29.151 INFO:teuthology.orchestra.run.smithi088.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2020-03-27T18:38:29.151 INFO:teuthology.orchestra.run.smithi088.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2020-03-27T18:38:29.151 INFO:teuthology.orchestra.run.smithi088.stdout:Build Host : kbuilder.bsys.centos.org 2020-03-27T18:38:29.151 INFO:teuthology.orchestra.run.smithi088.stdout:Relocations : (not relocatable) 2020-03-27T18:38:29.151 INFO:teuthology.orchestra.run.smithi088.stdout:Packager : CentOS BuildSystem 2020-03-27T18:38:29.152 INFO:teuthology.orchestra.run.smithi088.stdout:Vendor : CentOS 2020-03-27T18:38:29.152 INFO:teuthology.orchestra.run.smithi088.stdout:URL : http://www.gnu.org/software/grub/ 2020-03-27T18:38:29.152 INFO:teuthology.orchestra.run.smithi088.stdout:Summary : Support tools for GRUB. 2020-03-27T18:38:29.152 INFO:teuthology.orchestra.run.smithi088.stdout:Description : 2020-03-27T18:38:29.152 INFO:teuthology.orchestra.run.smithi088.stdout: 2020-03-27T18:38:29.152 INFO:teuthology.orchestra.run.smithi088.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2020-03-27T18:38:29.152 INFO:teuthology.orchestra.run.smithi088.stdout:customizable bootloader with modular architecture. It supports a rich 2020-03-27T18:38:29.165 INFO:teuthology.orchestra.run.smithi088.stdout:variety of kernel formats, file systems, computer architectures and 2020-03-27T18:38:29.166 INFO:teuthology.orchestra.run.smithi088.stdout:hardware devices. 2020-03-27T18:38:29.166 INFO:teuthology.orchestra.run.smithi088.stdout: 2020-03-27T18:38:29.166 INFO:teuthology.orchestra.run.smithi088.stdout:This subpackage provides tools for support of all platforms. 2020-03-27T18:38:29.166 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2020-03-27T18:38:29.166 INFO:teuthology.task.kernel:Updating grub on smithi088 to boot 3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:38:29.166 INFO:teuthology.orchestra.run.smithi088:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2020-03-27T18:38:29.610 INFO:teuthology.orchestra.run.smithi088.stderr:Generating grub configuration file ... 2020-03-27T18:38:29.745 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.18.1.el7.x86_64 2020-03-27T18:38:29.760 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.18.1.el7.x86_64.img 2020-03-27T18:38:29.985 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.12.1.el7.x86_64 2020-03-27T18:38:29.998 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.12.1.el7.x86_64.img 2020-03-27T18:38:30.026 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2020-03-27T18:38:30.043 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2020-03-27T18:38:30.068 INFO:teuthology.orchestra.run.smithi088.stderr:Found linux image: /boot/vmlinuz-0-rescue-11867a3080154962a87601c586b1198d 2020-03-27T18:38:30.076 INFO:teuthology.orchestra.run.smithi088.stderr:Found initrd image: /boot/initramfs-0-rescue-11867a3080154962a87601c586b1198d.img 2020-03-27T18:38:30.978 INFO:teuthology.orchestra.run.smithi088.stderr:done 2020-03-27T18:38:30.979 INFO:teuthology.orchestra.run.smithi088:> mktemp 2020-03-27T18:38:31.035 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.G7ylA7DWp6 2020-03-27T18:38:31.035 INFO:teuthology.orchestra.run.smithi088:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.G7ylA7DWp6 2020-03-27T18:38:31.108 INFO:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.G7ylA7DWp6 2020-03-27T18:38:31.249 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.G7ylA7DWp6 is 6KB 2020-03-27T18:38:31.323 INFO:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.G7ylA7DWp6 2020-03-27T18:38:31.394 INFO:teuthology.orchestra.run.smithi088:> sudo grub2-set-default 0 2020-03-27T18:38:31.626 INFO:teuthology.orchestra.run.smithi088:> sudo shutdown -r now 2020-03-27T18:39:01.657 INFO:teuthology.misc:Re-opening connections... 2020-03-27T18:39:01.657 INFO:teuthology.misc:trying to connect to ubuntu@smithi013.front.sepia.ceph.com 2020-03-27T18:39:01.657 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-03-27T18:39:01.658 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi013.front.sepia.ceph.com', 'timeout': 60} 2020-03-27T18:39:19.938 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2020-03-27T18:39:19.938 DEBUG:teuthology.task.console_log:Killing console logger for smithi175 2020-03-27T18:39:19.938 DEBUG:teuthology.task.console_log:Killing console logger for smithi013 2020-03-27T18:39:19.938 DEBUG:teuthology.task.console_log:Killing console logger for smithi088 2020-03-27T18:39:19.939 DEBUG:teuthology.task.console_log:Killing console logger for smithi175 2020-03-27T18:39:19.939 DEBUG:teuthology.task.console_log:Killing console logger for smithi013 2020-03-27T18:39:19.939 DEBUG:teuthology.task.console_log:Killing console logger for smithi088 2020-03-27T18:39:19.940 DEBUG:teuthology.exit:Finished running handlers