2019-11-28T14:29:30.296 INFO:root:teuthology version: 1.0.0-e749e7b 2019-11-28T14:29:30.297 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.4pg6Ao.tmp -- --verbose --lock --description rados:mgr/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/filestore-xfs.yaml supported-random-distro$/{centos_7.yaml} tasks/prometheus.yaml} --name laura-2019-11-28_10:11:08-rados:mgr-wip-laura-testing-30961-distro-basic-mira --block --owner scheduled_laura@teuthology --archive /home/teuthworker/archive/laura-2019-11-28_10:11:08-rados:mgr-wip-laura-testing-30961-distro-basic-mira/4550044 2019-11-28T14:29:30.334 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-11-28T14:29:30.386 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/laura-2019-11-28_10:11:08-rados:mgr-wip-laura-testing-30961-distro-basic-mira/4550044 branch: wip-laura-testing-30961 description: rados:mgr/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/filestore-xfs.yaml supported-random-distro$/{centos_7.yaml} tasks/prometheus.yaml} email: lpaduano@suse.com first_in_suite: false job_id: '4550044' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: mira name: laura-2019-11-28_10:11:08-rados:mgr-wip-laura-testing-30961-distro-basic-mira nuke-on-error: true openstack: - volumes: count: 2 size: 30 os_type: centos os_version: '7.6' overrides: admin_socket: branch: wip-laura-testing-30961 ceph: conf: client: debug client: 20 debug mgrc: 20 debug ms: 1 mds: debug mgrc: 20 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug filestore: 20 debug journal: 20 debug mgrc: 20 debug ms: 1 debug osd: 25 osd objectstore: filestore osd sloppy crc: true fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: ee6e3d1118793f0aa77634c532f4652b8cdfde39 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: osd objectstore: filestore osd sloppy crc: true filestore: true fs: xfs install: ceph: sha1: ee6e3d1118793f0aa77634c532f4652b8cdfde39 workunit: branch: wip-laura-testing-30961 sha1: ee6e3d1118793f0aa77634c532f4652b8cdfde39 owner: scheduled_laura@teuthology priority: 75 repo: git://git.ceph.com/ceph-ci.git roles: - - mgr.x - mon.a - mon.c - mds.a - mds.c - osd.0 - client.0 - - mgr.y - mgr.z - mon.b - mds.b - osd.1 - osd.2 - osd.3 - client.1 sha1: ee6e3d1118793f0aa77634c532f4652b8cdfde39 suite: rados:mgr suite_branch: wip-laura-testing-30961 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-laura-testing-30961/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: ee6e3d1118793f0aa77634c532f4652b8cdfde39 tasks: - install: null - ceph: log-whitelist: - overall HEALTH_ - \(MGR_DOWN\) - \(PG_ - replacing it with standby - No standby daemons available wait-for-scrub: false - cephfs_test_runner: modules: - tasks.mgr.test_prometheus teuthology_branch: master tube: mira verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.mira.9300 2019-11-28T14:29:30.386 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-laura-testing-30961/qa; will attempt to use it 2019-11-28T14:29:30.388 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-laura-testing-30961/qa/tasks 2019-11-28T14:29:30.388 INFO:teuthology.run_tasks:Running task internal.check_packages... 2019-11-28T14:29:30.390 INFO:teuthology.task.internal:Checking packages... 2019-11-28T14:29:30.403 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash 'ee6e3d1118793f0aa77634c532f4652b8cdfde39' 2019-11-28T14:29:30.403 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-11-28T14:29:30.403 INFO:teuthology.packaging:ref: None 2019-11-28T14:29:30.403 INFO:teuthology.packaging:tag: None 2019-11-28T14:29:30.404 INFO:teuthology.packaging:branch: wip-laura-testing-30961 2019-11-28T14:29:30.404 INFO:teuthology.packaging:sha1: ee6e3d1118793f0aa77634c532f4652b8cdfde39 2019-11-28T14:29:30.404 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&ref=wip-laura-testing-30961 2019-11-28T14:29:30.569 INFO:teuthology.task.internal:Found packages for ceph version 15.0.0-7797.gee6e3d1 2019-11-28T14:29:30.570 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2019-11-28T14:29:32.727 INFO:teuthology.task.internal:no buildpackages task found 2019-11-28T14:29:32.727 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2019-11-28T14:29:32.739 INFO:teuthology.task.internal.lock_machines:Locking machines... 2019-11-28T14:29:32.739 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-11-28T14:29:32.788 DEBUG:teuthology.lock.ops:lock_many request: {'count': 2, 'locked_by': 'scheduled_laura@teuthology', 'description': '/home/teuthworker/archive/laura-2019-11-28_10:11:08-rados:mgr-wip-laura-testing-30961-distro-basic-mira/4550044', 'machine_type': 'mira'} 2019-11-28T14:29:32.804 DEBUG:teuthology.lock.ops:locked ubuntu@mira063.front.sepia.ceph.com, ubuntu@mira066.front.sepia.ceph.com 2019-11-28T14:29:32.948 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-11-28T14:29:33.147 INFO:teuthology.provision.fog.mira066:Scheduling deploy of centos 7.6 2019-11-28T14:29:33.182 INFO:teuthology.provision.fog.mira063:Scheduling deploy of centos 7.6 2019-11-28T14:29:33.651 INFO:teuthology.orchestra.console:Power off mira066 2019-11-28T14:29:33.652 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-11-28T14:29:33.695 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-28T14:29:33.753 INFO:teuthology.orchestra.console:Power off mira063 2019-11-28T14:29:33.753 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-11-28T14:29:33.799 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-28T14:29:37.784 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-28T14:29:37.847 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-28T14:29:41.916 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-28T14:29:41.977 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-28T14:29:42.147 INFO:teuthology.orchestra.console:Power off for mira066 completed 2019-11-28T14:29:42.206 INFO:teuthology.orchestra.console:Power off for mira063 completed 2019-11-28T14:29:42.248 INFO:teuthology.orchestra.console:Power on mira066 2019-11-28T14:29:42.249 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-11-28T14:29:42.292 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-28T14:29:42.320 INFO:teuthology.orchestra.console:Power on mira063 2019-11-28T14:29:42.320 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-11-28T14:29:42.364 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-28T14:29:46.352 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-28T14:29:46.411 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-28T14:29:50.481 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-28T14:29:50.542 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H mira063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-28T14:29:50.709 INFO:teuthology.orchestra.console:Power on for mira066 completed 2019-11-28T14:29:50.774 INFO:teuthology.orchestra.console:Power on for mira063 completed 2019-11-28T14:29:50.811 INFO:teuthology.provision.fog.mira066:Waiting for deploy to finish 2019-11-28T14:29:50.876 INFO:teuthology.provision.fog.mira063:Waiting for deploy to finish 2019-11-28T14:32:22.265 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira063.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:32:31.345 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira063.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:32:37.386 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira066.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:32:40.425 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira063.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:32:46.444 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira066.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:32:49.482 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira063.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:32:55.500 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira066.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:32:58.542 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira063.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:33:04.555 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira066.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:33:07.594 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira063.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:33:13.614 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira066.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:33:22.701 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira066.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:33:31.761 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira066.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:33:40.851 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira066.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:34:13.622 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira063.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:34:14.329 INFO:teuthology.orchestra.run.mira063:> hostname 2019-11-28T14:34:15.790 INFO:teuthology.orchestra.run.mira063.stdout:mira063.front.sepia.ceph.com 2019-11-28T14:34:15.791 INFO:teuthology.orchestra.run.mira063:> grep mira063.front.sepia.ceph.com /etc/hosts 2019-11-28T14:34:15.911 INFO:teuthology.orchestra.run.mira063.stdout:172.21.6.124 mira063.front.sepia.ceph.com mira063 2019-11-28T14:34:15.912 INFO:teuthology.orchestra.run.mira063:> sudo hostname mira063 2019-11-28T14:34:16.410 INFO:teuthology.orchestra.run.mira063:> sudo sed -i -e 's/mira063.front.sepia.ceph.com/mira063/g' /etc/hosts 2019-11-28T14:34:16.587 INFO:teuthology.orchestra.run.mira063:> sudo sed -i -e 's/172.21.6.124/172.21.6.124/g' /etc/hosts 2019-11-28T14:34:16.749 INFO:teuthology.orchestra.run.mira063:> sudo sed -i -e 's/mira063.front.sepia.ceph.com/mira063/g' /etc/hostname 2019-11-28T14:34:16.904 INFO:teuthology.orchestra.run.mira063:> sudo hostname mira063 2019-11-28T14:34:17.061 INFO:teuthology.provision.fog.mira063:Deploy complete! 2019-11-28T14:34:17.071 DEBUG:teuthology.parallel:result is None 2019-11-28T14:34:21.361 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira066.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:34:21.666 INFO:teuthology.orchestra.run.mira066:> hostname 2019-11-28T14:34:22.366 INFO:teuthology.orchestra.run.mira066.stdout:mira066 2019-11-28T14:34:22.366 INFO:teuthology.orchestra.run.mira066:> grep mira066 /etc/hosts 2019-11-28T14:34:22.493 INFO:teuthology.orchestra.run.mira066.stdout:172.21.6.116 mira066.front.sepia.ceph.com mira066 2019-11-28T14:34:22.493 INFO:teuthology.orchestra.run.mira066:> sudo hostname mira066 2019-11-28T14:34:22.739 INFO:teuthology.orchestra.run.mira066:> sudo sed -i -e 's/mira066/mira066/g' /etc/hosts 2019-11-28T14:34:22.908 INFO:teuthology.orchestra.run.mira066:> sudo sed -i -e 's/172.21.6.116/172.21.6.116/g' /etc/hosts 2019-11-28T14:34:23.043 INFO:teuthology.orchestra.run.mira066:> sudo sed -i -e 's/mira066/mira066/g' /etc/hostname 2019-11-28T14:34:23.202 INFO:teuthology.orchestra.run.mira066:> sudo hostname mira066 2019-11-28T14:34:23.371 INFO:teuthology.provision.fog.mira066:Deploy complete! 2019-11-28T14:34:23.373 DEBUG:teuthology.parallel:result is None 2019-11-28T14:34:23.617 INFO:teuthology.lock.ops:Checking mira063.front.sepia.ceph.com 2019-11-28T14:34:23.618 INFO:teuthology.lock.ops:New key found. Updating... 2019-11-28T14:34:23.633 INFO:teuthology.lock.ops:Checking mira066.front.sepia.ceph.com 2019-11-28T14:34:23.634 INFO:teuthology.lock.ops:New key found. Updating... 2019-11-28T14:34:23.655 INFO:teuthology.lock.ops:Updating [mira063.front.sepia.ceph.com]: set os type and version on server 2019-11-28T14:34:23.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-28T14:34:23.656 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira063.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:34:23.804 INFO:teuthology.orchestra.run.mira063:> true 2019-11-28T14:34:24.215 INFO:teuthology.orchestra.run.mira063:> uname -m 2019-11-28T14:34:24.281 INFO:teuthology.orchestra.run.mira063.stdout:x86_64 2019-11-28T14:34:24.282 INFO:teuthology.orchestra.run.mira063:> cat /etc/os-release 2019-11-28T14:34:24.339 INFO:teuthology.orchestra.run.mira063.stdout:NAME="CentOS Linux" 2019-11-28T14:34:24.340 INFO:teuthology.orchestra.run.mira063.stdout:VERSION="7 (Core)" 2019-11-28T14:34:24.340 INFO:teuthology.orchestra.run.mira063.stdout:ID="centos" 2019-11-28T14:34:24.340 INFO:teuthology.orchestra.run.mira063.stdout:ID_LIKE="rhel fedora" 2019-11-28T14:34:24.340 INFO:teuthology.orchestra.run.mira063.stdout:VERSION_ID="7" 2019-11-28T14:34:24.340 INFO:teuthology.orchestra.run.mira063.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-11-28T14:34:24.340 INFO:teuthology.orchestra.run.mira063.stdout:ANSI_COLOR="0;31" 2019-11-28T14:34:24.341 INFO:teuthology.orchestra.run.mira063.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-11-28T14:34:24.341 INFO:teuthology.orchestra.run.mira063.stdout:HOME_URL="https://www.centos.org/" 2019-11-28T14:34:24.341 INFO:teuthology.orchestra.run.mira063.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-11-28T14:34:24.341 INFO:teuthology.orchestra.run.mira063.stdout: 2019-11-28T14:34:24.341 INFO:teuthology.orchestra.run.mira063.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-11-28T14:34:24.341 INFO:teuthology.orchestra.run.mira063.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-11-28T14:34:24.341 INFO:teuthology.orchestra.run.mira063.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-11-28T14:34:24.342 INFO:teuthology.orchestra.run.mira063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-11-28T14:34:24.342 INFO:teuthology.orchestra.run.mira063.stdout: 2019-11-28T14:34:24.343 INFO:teuthology.lock.ops:Updating mira063.front.sepia.ceph.com on lock server 2019-11-28T14:34:24.357 INFO:teuthology.lock.ops:Updating [mira066.front.sepia.ceph.com]: set os type and version on server 2019-11-28T14:34:24.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-28T14:34:24.358 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira066.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:34:24.462 INFO:teuthology.orchestra.run.mira066:> true 2019-11-28T14:34:24.828 INFO:teuthology.orchestra.run.mira066:> uname -m 2019-11-28T14:34:24.939 INFO:teuthology.orchestra.run.mira066.stdout:x86_64 2019-11-28T14:34:24.939 INFO:teuthology.orchestra.run.mira066:> cat /etc/os-release 2019-11-28T14:34:25.052 INFO:teuthology.orchestra.run.mira066.stdout:NAME="CentOS Linux" 2019-11-28T14:34:25.053 INFO:teuthology.orchestra.run.mira066.stdout:VERSION="7 (Core)" 2019-11-28T14:34:25.053 INFO:teuthology.orchestra.run.mira066.stdout:ID="centos" 2019-11-28T14:34:25.053 INFO:teuthology.orchestra.run.mira066.stdout:ID_LIKE="rhel fedora" 2019-11-28T14:34:25.053 INFO:teuthology.orchestra.run.mira066.stdout:VERSION_ID="7" 2019-11-28T14:34:25.053 INFO:teuthology.orchestra.run.mira066.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-11-28T14:34:25.053 INFO:teuthology.orchestra.run.mira066.stdout:ANSI_COLOR="0;31" 2019-11-28T14:34:25.053 INFO:teuthology.orchestra.run.mira066.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-11-28T14:34:25.054 INFO:teuthology.orchestra.run.mira066.stdout:HOME_URL="https://www.centos.org/" 2019-11-28T14:34:25.054 INFO:teuthology.orchestra.run.mira066.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-11-28T14:34:25.054 INFO:teuthology.orchestra.run.mira066.stdout: 2019-11-28T14:34:25.054 INFO:teuthology.orchestra.run.mira066.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-11-28T14:34:25.054 INFO:teuthology.orchestra.run.mira066.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-11-28T14:34:25.054 INFO:teuthology.orchestra.run.mira066.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-11-28T14:34:25.054 INFO:teuthology.orchestra.run.mira066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-11-28T14:34:25.055 INFO:teuthology.orchestra.run.mira066.stdout: 2019-11-28T14:34:25.055 INFO:teuthology.lock.ops:Updating mira066.front.sepia.ceph.com on lock server 2019-11-28T14:34:25.083 INFO:teuthology.task.internal.lock_machines:2 mira machines locked this try, 2/2 locked so far 2019-11-28T14:34:25.128 INFO:teuthology.task.internal.lock_machines:Locked targets: mira063.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoe6UeN5P0WBRP6NfNbgUW/UI1rw5uDRONowU9KurcKNlpT317u2KUidGmtw9JLw1BLAm6IPUVi9Gt5s3JNvVkpQwCJrdI3TIES55IQkIYoxNcilJ8sRMdI/F0WSqmf+8F3+UtZcFge0CFyiQa31VLg9gTxNxHBs0maQQ+slk6idUjV9iDYuBbpnbqj3o7iFxujXxSj0SLUdV4JgIRfQGU7MyYzRU0efgJXbNt62x/yPx0V6er3mUP9TvFlW+o+0hW5n5mOhjKI8KRzgX5CRnAsMojBy+ORYyFBs/iD4AlGlQ/DiJ9KlWWajkiRP+UhyZwRJKBLyOJoAi6WLe2RGtR mira066.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDK0ADVamtvYGnFm9g3KZ9RdiqSYJvttQB7ZeSo6/+TE5IHA450IQTSwwj4J+/GC9DMT+TKb0SZdzdAeA4N9v1B4+g3nIqOWcOJXbhS7zcEyJKqK8z67YrLFC7j977R8DaVpeZ0KxrlEtIh6WkGloc5c3FLnNFCmPVtKxZou04Xxiw6WRXemp+r4d1ixry3Xrom41jFKYXno9UupTrKAmDUGfJIUmrTIts8+40PqZuM9/FJKKGMStszjSDS/8h1U6az1a/8JBTNT/qm0+a7XFGiSeSiv/NT1/amDtCdcbuBJtRYbqPWeoq7ad8fqq0zO4Cp0b1+IpCbDCeNgJu3LGRr 2019-11-28T14:34:25.129 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-11-28T14:34:25.169 INFO:teuthology.run_tasks:Running task internal.save_config... 2019-11-28T14:34:25.180 INFO:teuthology.task.internal:Saving configuration 2019-11-28T14:34:25.195 INFO:teuthology.run_tasks:Running task internal.check_lock... 2019-11-28T14:34:25.203 INFO:teuthology.task.internal.check_lock:Checking locks... 2019-11-28T14:34:25.217 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-11-28 14:29:32.794093', u'locked_by': u'scheduled_laura@teuthology', u'up': True, u'mac_address': u'00:25:90:32:bf:28', u'name': u'mira063.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'mira', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKRwcfzWUFJ2HNcxuzNZ5Tav2G9P/YhY6P/ce0I4F9h1CaMMGMwDvgTwTrh+4p/Qmi+qtuvjTky/2lZ/moF0EkQ=', u'description': u'/home/teuthworker/archive/laura-2019-11-28_10:11:08-rados:mgr-wip-laura-testing-30961-distro-basic-mira/4550044'} 2019-11-28T14:34:25.230 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-11-28 14:29:32.794053', u'locked_by': u'scheduled_laura@teuthology', u'up': True, u'mac_address': u'00:25:90:0b:36:fa', u'name': u'mira066.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'mira', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBhb1MdZ3l+FVqRAV1rgiB5w3m6U77GDETyVb5VXRvP5qVBmNQJeFNTPdrf/M9dK/5S/bSiMI4eqIgZhjqYTQCg=', u'description': u'/home/teuthworker/archive/laura-2019-11-28_10:11:08-rados:mgr-wip-laura-testing-30961-distro-basic-mira/4550044'} 2019-11-28T14:34:25.231 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2019-11-28T14:34:25.239 INFO:teuthology.task.internal:roles: ubuntu@mira063.front.sepia.ceph.com - ['mgr.x', 'mon.a', 'mon.c', 'mds.a', 'mds.c', 'osd.0', 'client.0'] 2019-11-28T14:34:25.239 INFO:teuthology.task.internal:roles: ubuntu@mira066.front.sepia.ceph.com - ['mgr.y', 'mgr.z', 'mon.b', 'mds.b', 'osd.1', 'osd.2', 'osd.3', 'client.1'] 2019-11-28T14:34:25.240 INFO:teuthology.run_tasks:Running task console_log... 2019-11-28T14:34:25.408 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-11-28T14:34:25.408 INFO:teuthology.run_tasks:Running task internal.connect... 2019-11-28T14:34:25.420 INFO:teuthology.task.internal:Opening connections... 2019-11-28T14:34:25.420 DEBUG:teuthology.task.internal:connecting to ubuntu@mira063.front.sepia.ceph.com 2019-11-28T14:34:25.422 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira063.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:34:25.527 DEBUG:teuthology.task.internal:connecting to ubuntu@mira066.front.sepia.ceph.com 2019-11-28T14:34:25.528 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira066.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:34:25.672 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2019-11-28T14:34:25.681 INFO:teuthology.orchestra.run.mira063:> uname -m 2019-11-28T14:34:25.984 INFO:teuthology.orchestra.run.mira063.stdout:x86_64 2019-11-28T14:34:25.985 INFO:teuthology.orchestra.run.mira063:> cat /etc/os-release 2019-11-28T14:34:26.102 INFO:teuthology.orchestra.run.mira063.stdout:NAME="CentOS Linux" 2019-11-28T14:34:26.103 INFO:teuthology.orchestra.run.mira063.stdout:VERSION="7 (Core)" 2019-11-28T14:34:26.103 INFO:teuthology.orchestra.run.mira063.stdout:ID="centos" 2019-11-28T14:34:26.103 INFO:teuthology.orchestra.run.mira063.stdout:ID_LIKE="rhel fedora" 2019-11-28T14:34:26.103 INFO:teuthology.orchestra.run.mira063.stdout:VERSION_ID="7" 2019-11-28T14:34:26.104 INFO:teuthology.orchestra.run.mira063.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-11-28T14:34:26.104 INFO:teuthology.orchestra.run.mira063.stdout:ANSI_COLOR="0;31" 2019-11-28T14:34:26.104 INFO:teuthology.orchestra.run.mira063.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-11-28T14:34:26.104 INFO:teuthology.orchestra.run.mira063.stdout:HOME_URL="https://www.centos.org/" 2019-11-28T14:34:26.104 INFO:teuthology.orchestra.run.mira063.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-11-28T14:34:26.104 INFO:teuthology.orchestra.run.mira063.stdout: 2019-11-28T14:34:26.105 INFO:teuthology.orchestra.run.mira063.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-11-28T14:34:26.105 INFO:teuthology.orchestra.run.mira063.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-11-28T14:34:26.105 INFO:teuthology.orchestra.run.mira063.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-11-28T14:34:26.105 INFO:teuthology.orchestra.run.mira063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-11-28T14:34:26.105 INFO:teuthology.orchestra.run.mira063.stdout: 2019-11-28T14:34:26.105 INFO:teuthology.lock.ops:Updating mira063.front.sepia.ceph.com on lock server 2019-11-28T14:34:26.117 INFO:teuthology.orchestra.run.mira066:> uname -m 2019-11-28T14:34:26.211 INFO:teuthology.orchestra.run.mira066.stdout:x86_64 2019-11-28T14:34:26.211 INFO:teuthology.orchestra.run.mira066:> cat /etc/os-release 2019-11-28T14:34:26.275 INFO:teuthology.orchestra.run.mira066.stdout:NAME="CentOS Linux" 2019-11-28T14:34:26.275 INFO:teuthology.orchestra.run.mira066.stdout:VERSION="7 (Core)" 2019-11-28T14:34:26.275 INFO:teuthology.orchestra.run.mira066.stdout:ID="centos" 2019-11-28T14:34:26.275 INFO:teuthology.orchestra.run.mira066.stdout:ID_LIKE="rhel fedora" 2019-11-28T14:34:26.275 INFO:teuthology.orchestra.run.mira066.stdout:VERSION_ID="7" 2019-11-28T14:34:26.276 INFO:teuthology.orchestra.run.mira066.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-11-28T14:34:26.276 INFO:teuthology.orchestra.run.mira066.stdout:ANSI_COLOR="0;31" 2019-11-28T14:34:26.276 INFO:teuthology.orchestra.run.mira066.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-11-28T14:34:26.276 INFO:teuthology.orchestra.run.mira066.stdout:HOME_URL="https://www.centos.org/" 2019-11-28T14:34:26.276 INFO:teuthology.orchestra.run.mira066.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-11-28T14:34:26.276 INFO:teuthology.orchestra.run.mira066.stdout: 2019-11-28T14:34:26.276 INFO:teuthology.orchestra.run.mira066.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-11-28T14:34:26.277 INFO:teuthology.orchestra.run.mira066.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-11-28T14:34:26.277 INFO:teuthology.orchestra.run.mira066.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-11-28T14:34:26.277 INFO:teuthology.orchestra.run.mira066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-11-28T14:34:26.277 INFO:teuthology.orchestra.run.mira066.stdout: 2019-11-28T14:34:26.278 INFO:teuthology.lock.ops:Updating mira066.front.sepia.ceph.com on lock server 2019-11-28T14:34:26.289 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2019-11-28T14:34:26.340 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2019-11-28T14:34:26.351 INFO:teuthology.task.internal:Checking for old test directory... 2019-11-28T14:34:26.352 INFO:teuthology.orchestra.run.mira063:> test '!' -e /home/ubuntu/cephtest 2019-11-28T14:34:26.368 INFO:teuthology.orchestra.run.mira066:> test '!' -e /home/ubuntu/cephtest 2019-11-28T14:34:26.470 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2019-11-28T14:34:26.484 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2019-11-28T14:34:26.484 INFO:teuthology.orchestra.run.mira063:> test -z $(ls -A /var/lib/ceph) 2019-11-28T14:34:26.490 INFO:teuthology.orchestra.run.mira066:> test -z $(ls -A /var/lib/ceph) 2019-11-28T14:34:26.582 INFO:teuthology.orchestra.run.mira066.stderr:ls: cannot access /var/lib/ceph: No such file or directory 2019-11-28T14:34:26.595 INFO:teuthology.orchestra.run.mira063.stderr:ls: cannot access /var/lib/ceph: No such file or directory 2019-11-28T14:34:26.597 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2019-11-28T14:34:26.636 INFO:teuthology.run_tasks:Running task kernel... 2019-11-28T14:34:26.670 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.c': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}, 'mds.a': {'sha1': 'distro', 'kdb': True}, 'mds.c': {'sha1': 'distro', 'kdb': True}, 'mds.b': {'sha1': 'distro', 'kdb': True}, 'osd.1': {'sha1': 'distro', 'kdb': True}, 'osd.0': {'sha1': 'distro', 'kdb': True}, 'osd.3': {'sha1': 'distro', 'kdb': True}, 'client.0': {'sha1': 'distro', 'kdb': True}, 'osd.2': {'sha1': 'distro', 'kdb': True}, 'mgr.y': {'sha1': 'distro', 'kdb': True}, 'mgr.x': {'sha1': 'distro', 'kdb': True}, 'mgr.z': {'sha1': 'distro', 'kdb': True}, 'client.1': {'sha1': 'distro', 'kdb': True}} 2019-11-28T14:34:26.674 INFO:teuthology.task.kernel:config {'mgr.y': {'sha1': 'distro', 'kdb': True}, 'mgr.x': {'sha1': 'distro', 'kdb': True}}, timeout 300 2019-11-28T14:34:26.674 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@mira063.front.sepia.ceph.com 2019-11-28T14:34:26.675 INFO:teuthology.orchestra.run.mira063:> sudo package-cleanup -y --oldkernels 2019-11-28T14:34:31.166 INFO:teuthology.orchestra.run.mira063.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-28T14:34:41.310 INFO:teuthology.orchestra.run.mira063.stdout:No old kernels to remove 2019-11-28T14:34:41.335 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@mira066.front.sepia.ceph.com 2019-11-28T14:34:41.335 INFO:teuthology.orchestra.run.mira066:> sudo package-cleanup -y --oldkernels 2019-11-28T14:34:42.273 INFO:teuthology.orchestra.run.mira066.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-28T14:34:49.317 INFO:teuthology.orchestra.run.mira066.stdout:No old kernels to remove 2019-11-28T14:34:49.342 INFO:teuthology.orchestra.run.mira066:> uname -r 2019-11-28T14:34:49.458 INFO:teuthology.orchestra.run.mira066.stdout:3.10.0-957.21.3.el7.x86_64 2019-11-28T14:34:49.459 INFO:teuthology.task.kernel:Running kernel on mira066: 3.10.0-957.21.3.el7.x86_64 2019-11-28T14:34:49.459 INFO:teuthology.orchestra.run.mira066:> sudo yum install -y kernel 2019-11-28T14:34:49.978 INFO:teuthology.orchestra.run.mira066.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-28T14:34:50.051 INFO:teuthology.orchestra.run.mira066.stdout:Determining fastest mirrors 2019-11-28T14:34:50.325 INFO:teuthology.orchestra.run.mira066.stdout: * base: reflector.westga.edu 2019-11-28T14:34:50.325 INFO:teuthology.orchestra.run.mira066.stdout: * epel: ftp.linux.ncsu.edu 2019-11-28T14:34:50.326 INFO:teuthology.orchestra.run.mira066.stdout: * extras: linux.cc.lehigh.edu 2019-11-28T14:34:50.327 INFO:teuthology.orchestra.run.mira066.stdout: * updates: mirror.wdc1.us.leaseweb.net 2019-11-28T14:34:55.129 INFO:teuthology.orchestra.run.mira066.stdout:1 packages excluded due to repository priority protections 2019-11-28T14:34:55.771 INFO:teuthology.orchestra.run.mira066.stdout:Resolving Dependencies 2019-11-28T14:34:55.772 INFO:teuthology.orchestra.run.mira066.stdout:--> Running transaction check 2019-11-28T14:34:55.772 INFO:teuthology.orchestra.run.mira066.stdout:---> Package kernel.x86_64 0:3.10.0-1062.4.3.el7 will be installed 2019-11-28T14:34:56.243 INFO:teuthology.orchestra.run.mira066.stdout:--> Processing Dependency: linux-firmware >= 20190429-72 for package: kernel-3.10.0-1062.4.3.el7.x86_64 2019-11-28T14:34:56.357 INFO:teuthology.orchestra.run.mira066.stdout:--> Running transaction check 2019-11-28T14:34:56.358 INFO:teuthology.orchestra.run.mira066.stdout:---> Package linux-firmware.noarch 0:20180911-69.git85c5d90.el7 will be updated 2019-11-28T14:34:56.359 INFO:teuthology.orchestra.run.mira066.stdout:---> Package linux-firmware.noarch 0:20190429-72.gitddde598.el7 will be an update 2019-11-28T14:34:57.660 INFO:teuthology.orchestra.run.mira066.stdout:--> Finished Dependency Resolution 2019-11-28T14:34:58.418 INFO:teuthology.orchestra.run.mira066.stdout: 2019-11-28T14:34:58.419 INFO:teuthology.orchestra.run.mira066.stdout:Dependencies Resolved 2019-11-28T14:34:58.432 INFO:teuthology.orchestra.run.mira066.stdout: 2019-11-28T14:34:58.433 INFO:teuthology.orchestra.run.mira066.stdout:================================================================================ 2019-11-28T14:34:58.433 INFO:teuthology.orchestra.run.mira066.stdout: Package Arch Version Repository Size 2019-11-28T14:34:58.433 INFO:teuthology.orchestra.run.mira066.stdout:================================================================================ 2019-11-28T14:34:58.433 INFO:teuthology.orchestra.run.mira066.stdout:Installing: 2019-11-28T14:34:58.434 INFO:teuthology.orchestra.run.mira066.stdout: kernel x86_64 3.10.0-1062.4.3.el7 updates 50 M 2019-11-28T14:34:58.434 INFO:teuthology.orchestra.run.mira066.stdout:Updating for dependencies: 2019-11-28T14:34:58.434 INFO:teuthology.orchestra.run.mira066.stdout: linux-firmware noarch 20190429-72.gitddde598.el7 base 73 M 2019-11-28T14:34:58.434 INFO:teuthology.orchestra.run.mira066.stdout: 2019-11-28T14:34:58.434 INFO:teuthology.orchestra.run.mira066.stdout:Transaction Summary 2019-11-28T14:34:58.434 INFO:teuthology.orchestra.run.mira066.stdout:================================================================================ 2019-11-28T14:34:58.434 INFO:teuthology.orchestra.run.mira066.stdout:Install 1 Package 2019-11-28T14:34:58.435 INFO:teuthology.orchestra.run.mira066.stdout:Upgrade ( 1 Dependent package) 2019-11-28T14:34:58.435 INFO:teuthology.orchestra.run.mira066.stdout: 2019-11-28T14:34:58.435 INFO:teuthology.orchestra.run.mira066.stdout:Total download size: 123 M 2019-11-28T14:34:58.435 INFO:teuthology.orchestra.run.mira066.stdout:Downloading packages: 2019-11-28T14:34:58.435 INFO:teuthology.orchestra.run.mira066.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-11-28T14:35:03.980 INFO:teuthology.orchestra.run.mira066.stdout:-------------------------------------------------------------------------------- 2019-11-28T14:35:03.981 INFO:teuthology.orchestra.run.mira066.stdout:Total 22 MB/s | 123 MB 00:05 2019-11-28T14:35:04.093 INFO:teuthology.orchestra.run.mira066.stdout:Running transaction check 2019-11-28T14:35:04.434 INFO:teuthology.orchestra.run.mira066.stdout:Running transaction test 2019-11-28T14:35:05.750 INFO:teuthology.orchestra.run.mira066.stdout:Transaction test succeeded 2019-11-28T14:35:05.751 INFO:teuthology.orchestra.run.mira066.stdout:Running transaction 2019-11-28T14:35:23.081 INFO:teuthology.orchestra.run.mira066.stdout: Updating : linux-firmware-20190429-72.gitddde598.el7.noarch 1/3 2019-11-28T14:35:28.891 INFO:teuthology.orchestra.run.mira066.stdout: Installing : kernel-3.10.0-1062.4.3.el7.x86_64 2/3 2019-11-28T14:36:17.447 INFO:teuthology.orchestra.run.mira066.stdout: Cleanup : linux-firmware-20180911-69.git85c5d90.el7.noarch 3/3 2019-11-28T14:36:17.508 INFO:teuthology.orchestra.run.mira066.stdout: Verifying : kernel-3.10.0-1062.4.3.el7.x86_64 1/3 2019-11-28T14:36:17.528 INFO:teuthology.orchestra.run.mira066.stdout: Verifying : linux-firmware-20190429-72.gitddde598.el7.noarch 2/3 2019-11-28T14:36:17.717 INFO:teuthology.orchestra.run.mira066.stdout: Verifying : linux-firmware-20180911-69.git85c5d90.el7.noarch 3/3 2019-11-28T14:36:17.717 INFO:teuthology.orchestra.run.mira066.stdout: 2019-11-28T14:36:17.717 INFO:teuthology.orchestra.run.mira066.stdout:Installed: 2019-11-28T14:36:17.717 INFO:teuthology.orchestra.run.mira066.stdout: kernel.x86_64 0:3.10.0-1062.4.3.el7 2019-11-28T14:36:17.717 INFO:teuthology.orchestra.run.mira066.stdout: 2019-11-28T14:36:17.717 INFO:teuthology.orchestra.run.mira066.stdout:Dependency Updated: 2019-11-28T14:36:17.718 INFO:teuthology.orchestra.run.mira066.stdout: linux-firmware.noarch 0:20190429-72.gitddde598.el7 2019-11-28T14:36:17.718 INFO:teuthology.orchestra.run.mira066.stdout: 2019-11-28T14:36:17.718 INFO:teuthology.orchestra.run.mira066.stdout:Complete! 2019-11-28T14:36:17.836 INFO:teuthology.orchestra.run.mira066:> rpm -q kernel --last | head -n 1 2019-11-28T14:36:18.324 INFO:teuthology.orchestra.run.mira066.stdout:kernel-3.10.0-1062.4.3.el7.x86_64 Thu 28 Nov 2019 02:35:23 PM UTC 2019-11-28T14:36:18.327 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.4.3.el7.x86_64 2019-11-28T14:36:18.327 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-957.21.3.el7.x86_64 Expected: 3.10.0-1062.4.3.el7.x86_64 2019-11-28T14:36:18.327 INFO:teuthology.orchestra.run.mira063:> uname -r 2019-11-28T14:36:18.464 INFO:teuthology.orchestra.run.mira063.stdout:3.10.0-957.21.3.el7.x86_64 2019-11-28T14:36:18.464 INFO:teuthology.task.kernel:Running kernel on mira063: 3.10.0-957.21.3.el7.x86_64 2019-11-28T14:36:18.464 INFO:teuthology.orchestra.run.mira063:> sudo yum install -y kernel 2019-11-28T14:36:18.896 INFO:teuthology.orchestra.run.mira063.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-28T14:36:18.968 INFO:teuthology.orchestra.run.mira063.stdout:Determining fastest mirrors 2019-11-28T14:36:19.134 INFO:teuthology.orchestra.run.mira063.stdout: * base: mirror.rackspace.com 2019-11-28T14:36:19.135 INFO:teuthology.orchestra.run.mira063.stdout: * epel: ftp.linux.ncsu.edu 2019-11-28T14:36:19.136 INFO:teuthology.orchestra.run.mira063.stdout: * extras: linux.cc.lehigh.edu 2019-11-28T14:36:19.140 INFO:teuthology.orchestra.run.mira063.stdout: * updates: mirror.linux.duke.edu 2019-11-28T14:36:25.079 INFO:teuthology.orchestra.run.mira063.stdout:1 packages excluded due to repository priority protections 2019-11-28T14:36:25.721 INFO:teuthology.orchestra.run.mira063.stdout:Resolving Dependencies 2019-11-28T14:36:25.722 INFO:teuthology.orchestra.run.mira063.stdout:--> Running transaction check 2019-11-28T14:36:25.722 INFO:teuthology.orchestra.run.mira063.stdout:---> Package kernel.x86_64 0:3.10.0-1062.4.3.el7 will be installed 2019-11-28T14:36:26.831 INFO:teuthology.orchestra.run.mira063.stdout:--> Processing Dependency: linux-firmware >= 20190429-72 for package: kernel-3.10.0-1062.4.3.el7.x86_64 2019-11-28T14:36:27.207 INFO:teuthology.orchestra.run.mira063.stdout:--> Running transaction check 2019-11-28T14:36:27.207 INFO:teuthology.orchestra.run.mira063.stdout:---> Package linux-firmware.noarch 0:20180911-69.git85c5d90.el7 will be updated 2019-11-28T14:36:27.293 INFO:teuthology.orchestra.run.mira063.stdout:---> Package linux-firmware.noarch 0:20190429-72.gitddde598.el7 will be an update 2019-11-28T14:36:33.229 INFO:teuthology.orchestra.run.mira063.stdout:--> Finished Dependency Resolution 2019-11-28T14:36:36.569 INFO:teuthology.orchestra.run.mira063.stdout: 2019-11-28T14:36:36.569 INFO:teuthology.orchestra.run.mira063.stdout:Dependencies Resolved 2019-11-28T14:36:36.588 INFO:teuthology.orchestra.run.mira063.stdout: 2019-11-28T14:36:36.588 INFO:teuthology.orchestra.run.mira063.stdout:================================================================================ 2019-11-28T14:36:36.589 INFO:teuthology.orchestra.run.mira063.stdout: Package Arch Version Repository Size 2019-11-28T14:36:36.589 INFO:teuthology.orchestra.run.mira063.stdout:================================================================================ 2019-11-28T14:36:36.589 INFO:teuthology.orchestra.run.mira063.stdout:Installing: 2019-11-28T14:36:36.589 INFO:teuthology.orchestra.run.mira063.stdout: kernel x86_64 3.10.0-1062.4.3.el7 updates 50 M 2019-11-28T14:36:36.589 INFO:teuthology.orchestra.run.mira063.stdout:Updating for dependencies: 2019-11-28T14:36:36.589 INFO:teuthology.orchestra.run.mira063.stdout: linux-firmware noarch 20190429-72.gitddde598.el7 base 73 M 2019-11-28T14:36:36.589 INFO:teuthology.orchestra.run.mira063.stdout: 2019-11-28T14:36:36.590 INFO:teuthology.orchestra.run.mira063.stdout:Transaction Summary 2019-11-28T14:36:36.590 INFO:teuthology.orchestra.run.mira063.stdout:================================================================================ 2019-11-28T14:36:36.590 INFO:teuthology.orchestra.run.mira063.stdout:Install 1 Package 2019-11-28T14:36:36.590 INFO:teuthology.orchestra.run.mira063.stdout:Upgrade ( 1 Dependent package) 2019-11-28T14:36:36.590 INFO:teuthology.orchestra.run.mira063.stdout: 2019-11-28T14:36:36.590 INFO:teuthology.orchestra.run.mira063.stdout:Total download size: 123 M 2019-11-28T14:36:36.590 INFO:teuthology.orchestra.run.mira063.stdout:Downloading packages: 2019-11-28T14:36:36.591 INFO:teuthology.orchestra.run.mira063.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-11-28T14:36:39.647 INFO:teuthology.orchestra.run.mira063.stdout:-------------------------------------------------------------------------------- 2019-11-28T14:36:39.648 INFO:teuthology.orchestra.run.mira063.stdout:Total 40 MB/s | 123 MB 00:03 2019-11-28T14:36:39.759 INFO:teuthology.orchestra.run.mira063.stdout:Running transaction check 2019-11-28T14:36:40.089 INFO:teuthology.orchestra.run.mira063.stdout:Running transaction test 2019-11-28T14:36:44.909 INFO:teuthology.orchestra.run.mira063.stdout:Transaction test succeeded 2019-11-28T14:36:44.909 INFO:teuthology.orchestra.run.mira063.stdout:Running transaction 2019-11-28T14:37:14.668 INFO:teuthology.orchestra.run.mira063.stdout: Updating : linux-firmware-20190429-72.gitddde598.el7.noarch 1/3 2019-11-28T14:37:24.656 INFO:teuthology.orchestra.run.mira063.stdout: Installing : kernel-3.10.0-1062.4.3.el7.x86_64 2/3 2019-11-28T14:38:14.427 INFO:teuthology.orchestra.run.mira063.stdout: Cleanup : linux-firmware-20180911-69.git85c5d90.el7.noarch 3/3 2019-11-28T14:38:14.499 INFO:teuthology.orchestra.run.mira063.stdout: Verifying : kernel-3.10.0-1062.4.3.el7.x86_64 1/3 2019-11-28T14:38:14.515 INFO:teuthology.orchestra.run.mira063.stdout: Verifying : linux-firmware-20190429-72.gitddde598.el7.noarch 2/3 2019-11-28T14:38:14.694 INFO:teuthology.orchestra.run.mira063.stdout: Verifying : linux-firmware-20180911-69.git85c5d90.el7.noarch 3/3 2019-11-28T14:38:14.695 INFO:teuthology.orchestra.run.mira063.stdout: 2019-11-28T14:38:14.695 INFO:teuthology.orchestra.run.mira063.stdout:Installed: 2019-11-28T14:38:14.695 INFO:teuthology.orchestra.run.mira063.stdout: kernel.x86_64 0:3.10.0-1062.4.3.el7 2019-11-28T14:38:14.695 INFO:teuthology.orchestra.run.mira063.stdout: 2019-11-28T14:38:14.695 INFO:teuthology.orchestra.run.mira063.stdout:Dependency Updated: 2019-11-28T14:38:14.695 INFO:teuthology.orchestra.run.mira063.stdout: linux-firmware.noarch 0:20190429-72.gitddde598.el7 2019-11-28T14:38:14.695 INFO:teuthology.orchestra.run.mira063.stdout: 2019-11-28T14:38:14.696 INFO:teuthology.orchestra.run.mira063.stdout:Complete! 2019-11-28T14:38:14.810 INFO:teuthology.orchestra.run.mira063:> rpm -q kernel --last | head -n 1 2019-11-28T14:38:15.268 INFO:teuthology.orchestra.run.mira063.stdout:kernel-3.10.0-1062.4.3.el7.x86_64 Thu 28 Nov 2019 02:37:14 PM UTC 2019-11-28T14:38:15.271 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.4.3.el7.x86_64 2019-11-28T14:38:15.271 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-957.21.3.el7.x86_64 Expected: 3.10.0-1062.4.3.el7.x86_64 2019-11-28T14:38:15.272 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2019-11-28T14:38:15.272 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-11-28T14:38:15.272 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-11-28T14:38:15.272 INFO:teuthology.task.kernel:Installing distro kernel on mgr.y... 2019-11-28T14:38:15.272 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@mira066.front.sepia.ceph.com, path=None, version=distro) 2019-11-28T14:38:15.272 INFO:teuthology.orchestra.run.mira066:> rpm -q kernel --last | head -n 1 2019-11-28T14:38:15.757 INFO:teuthology.orchestra.run.mira066.stdout:kernel-3.10.0-1062.4.3.el7.x86_64 Thu 28 Nov 2019 02:35:23 PM UTC 2019-11-28T14:38:15.758 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.4.3.el7.x86_64 2019-11-28T14:38:15.758 INFO:teuthology.orchestra.run.mira066:> sudo rpm -qi grub2 2019-11-28T14:38:15.983 INFO:teuthology.orchestra.run.mira066.stdout:Name : grub2 2019-11-28T14:38:15.983 INFO:teuthology.orchestra.run.mira066.stdout:Epoch : 1 2019-11-28T14:38:15.984 INFO:teuthology.orchestra.run.mira066.stdout:Version : 2.02 2019-11-28T14:38:15.984 INFO:teuthology.orchestra.run.mira066.stdout:Release : 0.76.el7.centos 2019-11-28T14:38:15.984 INFO:teuthology.orchestra.run.mira066.stdout:Architecture: x86_64 2019-11-28T14:38:15.984 INFO:teuthology.orchestra.run.mira066.stdout:Install Date: Fri 12 Jul 2019 03:03:58 PM UTC 2019-11-28T14:38:15.984 INFO:teuthology.orchestra.run.mira066.stdout:Group : System Environment/Base 2019-11-28T14:38:15.985 INFO:teuthology.orchestra.run.mira066.stdout:Size : 0 2019-11-28T14:38:15.985 INFO:teuthology.orchestra.run.mira066.stdout:License : GPLv3+ 2019-11-28T14:38:15.985 INFO:teuthology.orchestra.run.mira066.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-11-28T14:38:15.985 INFO:teuthology.orchestra.run.mira066.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-11-28T14:38:15.986 INFO:teuthology.orchestra.run.mira066.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-11-28T14:38:15.986 INFO:teuthology.orchestra.run.mira066.stdout:Build Host : kbuilder.bsys.centos.org 2019-11-28T14:38:15.986 INFO:teuthology.orchestra.run.mira066.stdout:Relocations : (not relocatable) 2019-11-28T14:38:15.986 INFO:teuthology.orchestra.run.mira066.stdout:Packager : CentOS BuildSystem 2019-11-28T14:38:15.986 INFO:teuthology.orchestra.run.mira066.stdout:Vendor : CentOS 2019-11-28T14:38:15.987 INFO:teuthology.orchestra.run.mira066.stdout:URL : http://www.gnu.org/software/grub/ 2019-11-28T14:38:15.987 INFO:teuthology.orchestra.run.mira066.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-11-28T14:38:15.987 INFO:teuthology.orchestra.run.mira066.stdout:Description : 2019-11-28T14:38:15.988 INFO:teuthology.orchestra.run.mira066.stdout: 2019-11-28T14:38:15.988 INFO:teuthology.orchestra.run.mira066.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-11-28T14:38:15.988 INFO:teuthology.orchestra.run.mira066.stdout:customizable bootloader with modular architecture. It supports a rich 2019-11-28T14:38:15.989 INFO:teuthology.orchestra.run.mira066.stdout:variety of kernel formats, file systems, computer architectures and 2019-11-28T14:38:15.989 INFO:teuthology.orchestra.run.mira066.stdout:hardware devices. 2019-11-28T14:38:15.991 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-11-28T14:38:15.991 INFO:teuthology.task.kernel:Updating grub on mira066 to boot 3.10.0-1062.4.3.el7.x86_64 2019-11-28T14:38:15.991 INFO:teuthology.orchestra.run.mira066:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-11-28T14:38:16.797 INFO:teuthology.orchestra.run.mira066.stderr:Generating grub configuration file ... 2019-11-28T14:38:17.023 INFO:teuthology.orchestra.run.mira066.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.3.el7.x86_64 2019-11-28T14:38:17.051 INFO:teuthology.orchestra.run.mira066.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.3.el7.x86_64.img 2019-11-28T14:38:17.585 INFO:teuthology.orchestra.run.mira066.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.21.3.el7.x86_64 2019-11-28T14:38:17.616 INFO:teuthology.orchestra.run.mira066.stderr:Found initrd image: /boot/initramfs-3.10.0-957.21.3.el7.x86_64.img 2019-11-28T14:38:17.681 INFO:teuthology.orchestra.run.mira066.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-11-28T14:38:17.712 INFO:teuthology.orchestra.run.mira066.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-11-28T14:38:17.774 INFO:teuthology.orchestra.run.mira066.stderr:Found linux image: /boot/vmlinuz-0-rescue-ed19c4d1a216479b9bcb22a6fb70d4b0 2019-11-28T14:38:17.795 INFO:teuthology.orchestra.run.mira066.stderr:Found initrd image: /boot/initramfs-0-rescue-ed19c4d1a216479b9bcb22a6fb70d4b0.img 2019-11-28T14:38:19.472 INFO:teuthology.orchestra.run.mira066.stderr:done 2019-11-28T14:38:19.476 INFO:teuthology.orchestra.run.mira066:> mktemp 2019-11-28T14:38:19.597 INFO:teuthology.orchestra.run.mira066.stdout:/tmp/tmp.z066WtzkRh 2019-11-28T14:38:19.645 INFO:teuthology.orchestra.run.mira066:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.z066WtzkRh 2019-11-28T14:38:19.819 INFO:teuthology.orchestra.run.mira066:> sudo chmod 0666 /tmp/tmp.z066WtzkRh 2019-11-28T14:38:20.097 DEBUG:teuthology.orchestra.remote:mira066:/tmp/tmp.z066WtzkRh is 6KB 2019-11-28T14:38:20.222 INFO:teuthology.orchestra.run.mira066:> rm -fr /tmp/tmp.z066WtzkRh 2019-11-28T14:38:20.340 INFO:teuthology.orchestra.run.mira066:> sudo grub2-set-default 0 2019-11-28T14:38:20.536 INFO:teuthology.orchestra.run.mira066:> sudo shutdown -r now 2019-11-28T14:38:20.542 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2019-11-28T14:38:20.542 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@mira063.front.sepia.ceph.com, path=None, version=distro) 2019-11-28T14:38:20.542 INFO:teuthology.orchestra.run.mira063:> rpm -q kernel --last | head -n 1 2019-11-28T14:38:21.021 INFO:teuthology.orchestra.run.mira063.stdout:kernel-3.10.0-1062.4.3.el7.x86_64 Thu 28 Nov 2019 02:37:14 PM UTC 2019-11-28T14:38:21.024 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.4.3.el7.x86_64 2019-11-28T14:38:21.024 INFO:teuthology.orchestra.run.mira063:> sudo rpm -qi grub2 2019-11-28T14:38:21.235 INFO:teuthology.orchestra.run.mira063.stdout:Name : grub2 2019-11-28T14:38:21.236 INFO:teuthology.orchestra.run.mira063.stdout:Epoch : 1 2019-11-28T14:38:21.236 INFO:teuthology.orchestra.run.mira063.stdout:Version : 2.02 2019-11-28T14:38:21.236 INFO:teuthology.orchestra.run.mira063.stdout:Release : 0.76.el7.centos 2019-11-28T14:38:21.236 INFO:teuthology.orchestra.run.mira063.stdout:Architecture: x86_64 2019-11-28T14:38:21.236 INFO:teuthology.orchestra.run.mira063.stdout:Install Date: Fri 12 Jul 2019 03:03:58 PM UTC 2019-11-28T14:38:21.236 INFO:teuthology.orchestra.run.mira063.stdout:Group : System Environment/Base 2019-11-28T14:38:21.237 INFO:teuthology.orchestra.run.mira063.stdout:Size : 0 2019-11-28T14:38:21.237 INFO:teuthology.orchestra.run.mira063.stdout:License : GPLv3+ 2019-11-28T14:38:21.237 INFO:teuthology.orchestra.run.mira063.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-11-28T14:38:21.237 INFO:teuthology.orchestra.run.mira063.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-11-28T14:38:21.237 INFO:teuthology.orchestra.run.mira063.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-11-28T14:38:21.237 INFO:teuthology.orchestra.run.mira063.stdout:Build Host : kbuilder.bsys.centos.org 2019-11-28T14:38:21.237 INFO:teuthology.orchestra.run.mira063.stdout:Relocations : (not relocatable) 2019-11-28T14:38:21.238 INFO:teuthology.orchestra.run.mira063.stdout:Packager : CentOS BuildSystem 2019-11-28T14:38:21.238 INFO:teuthology.orchestra.run.mira063.stdout:Vendor : CentOS 2019-11-28T14:38:21.238 INFO:teuthology.orchestra.run.mira063.stdout:URL : http://www.gnu.org/software/grub/ 2019-11-28T14:38:21.238 INFO:teuthology.orchestra.run.mira063.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-11-28T14:38:21.238 INFO:teuthology.orchestra.run.mira063.stdout:Description : 2019-11-28T14:38:21.238 INFO:teuthology.orchestra.run.mira063.stdout: 2019-11-28T14:38:21.238 INFO:teuthology.orchestra.run.mira063.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-11-28T14:38:21.238 INFO:teuthology.orchestra.run.mira063.stdout:customizable bootloader with modular architecture. It supports a rich 2019-11-28T14:38:21.239 INFO:teuthology.orchestra.run.mira063.stdout:variety of kernel formats, file systems, computer architectures and 2019-11-28T14:38:21.239 INFO:teuthology.orchestra.run.mira063.stdout:hardware devices. 2019-11-28T14:38:21.240 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-11-28T14:38:21.240 INFO:teuthology.task.kernel:Updating grub on mira063 to boot 3.10.0-1062.4.3.el7.x86_64 2019-11-28T14:38:21.240 INFO:teuthology.orchestra.run.mira063:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-11-28T14:38:21.819 INFO:teuthology.orchestra.run.mira063.stderr:Generating grub configuration file ... 2019-11-28T14:38:22.004 INFO:teuthology.orchestra.run.mira063.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.3.el7.x86_64 2019-11-28T14:38:22.033 INFO:teuthology.orchestra.run.mira063.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.3.el7.x86_64.img 2019-11-28T14:38:22.555 INFO:teuthology.orchestra.run.mira063.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.21.3.el7.x86_64 2019-11-28T14:38:22.581 INFO:teuthology.orchestra.run.mira063.stderr:Found initrd image: /boot/initramfs-3.10.0-957.21.3.el7.x86_64.img 2019-11-28T14:38:22.651 INFO:teuthology.orchestra.run.mira063.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-11-28T14:38:22.680 INFO:teuthology.orchestra.run.mira063.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-11-28T14:38:22.742 INFO:teuthology.orchestra.run.mira063.stderr:Found linux image: /boot/vmlinuz-0-rescue-ed19c4d1a216479b9bcb22a6fb70d4b0 2019-11-28T14:38:22.771 INFO:teuthology.orchestra.run.mira063.stderr:Found initrd image: /boot/initramfs-0-rescue-ed19c4d1a216479b9bcb22a6fb70d4b0.img 2019-11-28T14:38:23.970 INFO:teuthology.orchestra.run.mira063.stderr:done 2019-11-28T14:38:23.973 INFO:teuthology.orchestra.run.mira063:> mktemp 2019-11-28T14:38:24.096 INFO:teuthology.orchestra.run.mira063.stdout:/tmp/tmp.OuHsl70vcc 2019-11-28T14:38:24.097 INFO:teuthology.orchestra.run.mira063:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.OuHsl70vcc 2019-11-28T14:38:24.260 INFO:teuthology.orchestra.run.mira063:> sudo chmod 0666 /tmp/tmp.OuHsl70vcc 2019-11-28T14:38:24.555 DEBUG:teuthology.orchestra.remote:mira063:/tmp/tmp.OuHsl70vcc is 6KB 2019-11-28T14:38:24.673 INFO:teuthology.orchestra.run.mira063:> rm -fr /tmp/tmp.OuHsl70vcc 2019-11-28T14:38:24.775 INFO:teuthology.orchestra.run.mira063:> sudo grub2-set-default 0 2019-11-28T14:38:24.959 INFO:teuthology.orchestra.run.mira063:> sudo shutdown -r now 2019-11-28T14:38:24.964 INFO:teuthology.misc:Re-opening connections... 2019-11-28T14:38:24.964 INFO:teuthology.misc:trying to connect to ubuntu@mira063.front.sepia.ceph.com 2019-11-28T14:38:24.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-28T14:38:24.967 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'mira063.front.sepia.ceph.com', 'timeout': 60} 2019-11-28T14:38:25.101 INFO:teuthology.orchestra.run.mira063:> true 2019-11-29T02:29:43.655 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2019-11-29T02:29:43.670 DEBUG:teuthology.task.console_log:Killing console logger for mira063 2019-11-29T02:29:43.671 DEBUG:teuthology.task.console_log:Killing console logger for mira066 2019-11-29T02:29:43.671 DEBUG:teuthology.task.console_log:Killing console logger for mira063 2019-11-29T02:29:43.671 DEBUG:teuthology.task.console_log:Killing console logger for mira066 2019-11-29T02:29:43.672 DEBUG:teuthology.exit:Finished running handlers