2019-11-08T18:42:56.476 INFO:root:teuthology version: 1.0.0-8e58544 2019-11-08T18:42:56.477 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.kkABjc.tmp -- --verbose --lock --description rados/standalone/{supported-random-distro$/{ubuntu_latest.yaml} workloads/scrub.yaml} --name sage-2019-11-08_17:49:46-rados-master-distro-basic-smithi --block --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2019-11-08_17:49:46-rados-master-distro-basic-smithi/4484650 2019-11-08T18:42:56.498 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-11-08T18:42:56.549 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sage-2019-11-08_17:49:46-rados-master-distro-basic-smithi/4484650 branch: master description: rados/standalone/{supported-random-distro$/{ubuntu_latest.yaml} workloads/scrub.yaml} email: null first_in_suite: false job_id: '4484650' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: sage-2019-11-08_17:49:46-rados-master-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: master ceph: conf: mon: debug mon: 20 debug ms: 20 debug paxos: 20 osd: debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 457b9560a30baf1e82c12db76889a7bd6b8d7a8e ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: 457b9560a30baf1e82c12db76889a7bd6b8d7a8e workunit: branch: master sha1: 25e26d27cf1d18c025a717b3adeaff12c8df41a4 owner: scheduled_sage@teuthology priority: 80 repo: git://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 sha1: 457b9560a30baf1e82c12db76889a7bd6b8d7a8e suite: rados suite_branch: master suite_path: /home/teuthworker/src/git.ceph.com_ceph_master/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph.git suite_sha1: 25e26d27cf1d18c025a717b3adeaff12c8df41a4 tasks: - install: null - workunit: basedir: qa/standalone clients: all: - scrub teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.23623 2019-11-08T18:42:56.550 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_master/qa; will attempt to use it 2019-11-08T18:42:56.550 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_master/qa/tasks 2019-11-08T18:42:56.550 INFO:teuthology.run_tasks:Running task internal.check_packages... 2019-11-08T18:42:56.552 INFO:teuthology.task.internal:Checking packages... 2019-11-08T18:42:56.565 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash '457b9560a30baf1e82c12db76889a7bd6b8d7a8e' 2019-11-08T18:42:56.565 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-11-08T18:42:56.565 INFO:teuthology.packaging:ref: None 2019-11-08T18:42:56.565 INFO:teuthology.packaging:tag: None 2019-11-08T18:42:56.565 INFO:teuthology.packaging:branch: master 2019-11-08T18:42:56.566 INFO:teuthology.packaging:sha1: 457b9560a30baf1e82c12db76889a7bd6b8d7a8e 2019-11-08T18:42:56.566 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=master 2019-11-08T18:42:57.477 INFO:teuthology.task.internal:Found packages for ceph version 15.0.0-6988-g25e26d2-1bionic 2019-11-08T18:42:57.477 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2019-11-08T18:42:57.544 INFO:teuthology.task.internal:no buildpackages task found 2019-11-08T18:42:57.544 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2019-11-08T18:42:57.667 INFO:teuthology.task.internal.lock_machines:Locking machines... 2019-11-08T18:42:57.668 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-11-08T18:42:57.710 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 0)... 2019-11-08T18:43:07.718 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 0)... 2019-11-08T18:43:17.735 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 0)... 2019-11-08T18:43:27.753 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 0)... 2019-11-08T18:43:37.770 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2019-11-08T18:43:47.782 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2019-11-08T18:43:57.792 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 1, have 2)... 2019-11-08T18:44:07.808 DEBUG:teuthology.lock.ops:lock_many request: {'count': 1, 'locked_by': 'scheduled_sage@teuthology', 'description': '/home/teuthworker/archive/sage-2019-11-08_17:49:46-rados-master-distro-basic-smithi/4484650', 'machine_type': 'smithi'} 2019-11-08T18:44:07.824 DEBUG:teuthology.lock.ops:locked ubuntu@smithi182.front.sepia.ceph.com 2019-11-08T18:44:07.887 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-11-08T18:44:08.083 INFO:teuthology.provision.fog.smithi182:Scheduling deploy of ubuntu 18.04 2019-11-08T18:44:08.392 INFO:teuthology.orchestra.console:Power off smithi182 2019-11-08T18:44:08.393 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-11-08T18:44:08.433 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-08T18:44:12.479 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-08T18:44:16.609 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-08T18:44:16.836 INFO:teuthology.orchestra.console:Power off for smithi182 completed 2019-11-08T18:44:16.937 INFO:teuthology.orchestra.console:Power on smithi182 2019-11-08T18:44:16.937 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-11-08T18:44:16.976 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-08T18:44:21.017 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-08T18:44:25.158 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-08T18:44:25.386 INFO:teuthology.orchestra.console:Power on for smithi182 completed 2019-11-08T18:44:25.487 INFO:teuthology.provision.fog.smithi182:Waiting for deploy to finish 2019-11-08T18:47:41.631 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:48:06.186 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:48:15.246 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:48:24.302 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:48:33.358 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:48:42.417 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:48:51.502 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:49:57.504 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:49:57.514 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2019-11-08T18:50:03.524 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:50:03.534 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2019-11-08T18:50:09.544 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:50:09.554 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2019-11-08T18:50:15.564 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:50:15.573 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2019-11-08T18:50:21.583 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:50:21.593 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2019-11-08T18:50:27.603 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:50:27.613 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2019-11-08T18:50:33.622 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:50:33.633 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2019-11-08T18:50:39.643 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:50:39.653 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2019-11-08T18:50:45.662 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:50:45.672 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2019-11-08T18:50:51.682 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:51:57.690 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:52:04.523 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:52:13.579 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:52:22.637 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:52:31.694 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:52:40.750 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:52:49.808 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:52:58.892 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:53:07.950 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:53:17.006 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:53:26.062 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:53:35.115 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:53:44.174 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:53:53.225 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:54:02.285 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:54:11.342 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:54:20.398 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:54:29.454 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:54:38.510 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:54:47.566 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:54:56.622 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:55:05.673 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:55:14.730 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:55:23.788 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:55:32.845 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:55:41.906 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:55:50.986 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:56:00.044 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:56:09.101 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:56:18.157 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:56:27.213 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:56:36.266 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:56:45.326 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:56:54.385 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:57:03.470 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:57:12.525 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:57:21.578 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:57:30.637 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:57:39.697 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:57:48.782 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:57:57.835 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:58:06.889 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:58:15.953 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:58:25.037 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:58:34.094 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:58:43.145 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:58:52.209 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:59:01.290 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:59:10.350 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:59:19.406 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:59:28.466 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:59:37.546 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:59:46.604 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T18:59:55.661 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:00:04.721 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:00:13.806 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:00:22.857 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:00:31.918 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:00:40.972 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:00:50.034 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:00:59.117 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:01:08.173 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:01:17.229 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:01:26.285 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:01:35.342 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:01:44.394 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:01:53.457 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:02:02.539 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:02:11.595 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:02:20.654 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:02:29.705 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:02:38.765 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:02:47.822 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:02:56.882 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:03:05.962 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:03:15.018 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:03:24.077 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:03:33.129 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:03:42.193 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:03:51.277 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:04:00.333 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:04:09.389 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:04:18.450 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi182.front.sepia.ceph.com', 'timeout': 60} 2019-11-08T19:04:21.529 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 89, in run_tasks manager.__enter__() File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/lock_machines.py", line 79, in lock_machines os_version, arch) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/lock/ops.py", line 145, in lock_many reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 85, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 99, in next resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 22, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/provision/__init__.py", line 26, in reimage return fog_obj.create() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/provision/fog.py", line 89, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/provision/fog.py", line 261, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 132, in __call__ raise MaxWhileTries(error_msg) MaxWhileTries: reached maximum tries (100) after waiting for 600 seconds 2019-11-08T19:04:21.530 DEBUG:teuthology.run_tasks:Unwinding manager internal.lock_machines 2019-11-08T19:04:21.601 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (100) after waiting for 600 seconds 2019-11-08T19:04:21.603 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest.yaml} workloads/scrub.yaml} failure_reason: reached maximum tries (100) after waiting for 600 seconds owner: scheduled_sage@teuthology status: dead success: false 2019-11-08T19:04:21.604 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-11-08T19:04:21.645 INFO:teuthology.run:DEAD