2020-08-31T22:09:22.154 INFO:root:teuthology version: 1.0.0-ec650ed 2020-08-31T22:09:22.156 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2020-08-31_21:26:19-kcephfs-wip-pdonnell-testing-20200829.172552-distro-basic-smithi/5396255 --lock --description kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_fsx} --block --name pdonnell-2020-08-31_21:26:19-kcephfs-wip-pdonnell-testing-20200829.172552-distro-basic-smithi --verbose --owner scheduled_pdonnell@teuthology -- /tmp/teuthology-worker.azpnt_bz.tmp 2020-08-31T22:09:22.194 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-08-31T22:09:22.256 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2020-08-31_21:26:19-kcephfs-wip-pdonnell-testing-20200829.172552-distro-basic-smithi/5396255 branch: wip-pdonnell-testing-20200829.172552 description: kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_fsx} email: pdonnell@redhat.com first_in_suite: false job_id: '5396255' kernel: client: branch: testing kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2020-08-31_21:26:19-kcephfs-wip-pdonnell-testing-20200829.172552-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: '8.1' overrides: admin_socket: branch: wip-pdonnell-testing-20200829.172552 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 osd objectstore: filestore osd op complaint time: 180 osd shutdown pgref assert: true osd sloppy crc: true fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: c5af0d54b89ffb9f4c441f7a4e930a89dee8fecf ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: osd objectstore: filestore osd sloppy crc: true filestore: true fs: xfs install: ceph: sha1: c5af0d54b89ffb9f4c441f7a4e930a89dee8fecf selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-pdonnell-testing-20200829.172552 sha1: c5af0d54b89ffb9f4c441f7a4e930a89dee8fecf owner: scheduled_pdonnell@teuthology priority: 75 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - - client.0 sha1: c5af0d54b89ffb9f4c441f7a4e930a89dee8fecf sleep_before_teardown: 0 suite: kcephfs suite_branch: wip-pdonnell-testing-20200829.172552 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200829.172552/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: c5af0d54b89ffb9f4c441f7a4e930a89dee8fecf tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - exec: client.0: - sudo ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - kclient: null - workunit: clients: all: - suites/fsx.sh teuthology_branch: refs/pull/1554/merge timestamp: 2020-08-31_21:26:19 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.29907 2020-08-31T22:09:22.256 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200829.172552/qa; will attempt to use it 2020-08-31T22:09:22.257 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200829.172552/qa/tasks 2020-08-31T22:09:22.258 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-08-31T22:09:22.259 INFO:teuthology.task.internal:Checking packages... 2020-08-31T22:09:22.282 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash 'c5af0d54b89ffb9f4c441f7a4e930a89dee8fecf' 2020-08-31T22:09:22.283 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-08-31T22:09:22.283 INFO:teuthology.packaging:ref: None 2020-08-31T22:09:22.284 INFO:teuthology.packaging:tag: None 2020-08-31T22:09:22.284 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20200829.172552 2020-08-31T22:09:22.284 INFO:teuthology.packaging:sha1: c5af0d54b89ffb9f4c441f7a4e930a89dee8fecf 2020-08-31T22:09:22.285 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20200829.172552 2020-08-31T22:09:22.456 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-4885.gc5af0d5 2020-08-31T22:09:22.457 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-08-31T22:09:22.472 INFO:teuthology.task.internal:no buildpackages task found 2020-08-31T22:09:22.472 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-08-31T22:09:22.497 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-08-31T22:09:22.498 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-08-31T22:09:22.555 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-08-31T22:09:32.574 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-08-31T22:09:42.592 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-08-31T22:09:52.613 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-08-31T22:10:02.632 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 1)... 2020-08-31T22:10:12.651 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-08-31T22:10:22.671 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-08-31T22:10:32.690 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 0)... 2020-08-31T22:10:42.713 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-08-31T22:10:52.731 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 3, have 2)... 2020-08-31T22:11:02.751 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_pdonnell@teuthology', 'count': 3, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/pdonnell-2020-08-31_21:26:19-kcephfs-wip-pdonnell-testing-20200829.172552-distro-basic-smithi/5396255'} 2020-08-31T22:11:02.779 DEBUG:teuthology.lock.ops:locked ubuntu@smithi026.front.sepia.ceph.com, ubuntu@smithi098.front.sepia.ceph.com, ubuntu@smithi187.front.sepia.ceph.com 2020-08-31T22:11:02.879 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fed53ae4d08>, signals=[15]) 2020-08-31T22:11:03.114 INFO:teuthology.provision.fog.smithi098:Scheduling deploy of centos 8.1 2020-08-31T22:11:03.118 INFO:teuthology.provision.fog.smithi187:Scheduling deploy of centos 8.1 2020-08-31T22:11:03.133 INFO:teuthology.provision.fog.smithi026:Scheduling deploy of centos 8.1 2020-08-31T22:11:03.537 INFO:teuthology.orchestra.console:Power off smithi187 2020-08-31T22:11:03.539 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-08-31T22:11:03.566 INFO:teuthology.orchestra.console:Power off smithi098 2020-08-31T22:11:03.567 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-08-31T22:11:03.586 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:03.607 INFO:teuthology.orchestra.console:Power off smithi026 2020-08-31T22:11:03.608 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-08-31T22:11:03.620 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:03.634 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:07.634 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:07.650 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:07.664 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:11.754 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:11.768 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:11.782 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:11.972 INFO:teuthology.orchestra.console:Power off for smithi187 completed 2020-08-31T22:11:11.986 INFO:teuthology.orchestra.console:Power off for smithi098 completed 2020-08-31T22:11:12.074 INFO:teuthology.orchestra.console:Power on smithi187 2020-08-31T22:11:12.075 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-08-31T22:11:12.088 INFO:teuthology.orchestra.console:Power on smithi098 2020-08-31T22:11:12.089 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-08-31T22:11:12.110 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:12.125 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:15.901 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:16.119 INFO:teuthology.orchestra.console:Power off for smithi026 completed 2020-08-31T22:11:16.141 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:16.155 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:16.221 INFO:teuthology.orchestra.console:Power on smithi026 2020-08-31T22:11:16.223 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-08-31T22:11:16.250 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:20.266 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:20.290 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:20.304 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:20.493 INFO:teuthology.orchestra.console:Power on for smithi187 completed 2020-08-31T22:11:20.507 INFO:teuthology.orchestra.console:Power on for smithi098 completed 2020-08-31T22:11:20.594 INFO:teuthology.provision.fog.smithi187:Waiting for deploy to finish 2020-08-31T22:11:20.609 INFO:teuthology.provision.fog.smithi098:Waiting for deploy to finish 2020-08-31T22:11:24.422 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-31T22:11:24.644 INFO:teuthology.orchestra.console:Power on for smithi026 completed 2020-08-31T22:11:24.745 INFO:teuthology.provision.fog.smithi026:Waiting for deploy to finish 2020-08-31T22:13:41.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:13:50.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:13:52.572 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:13:52.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:13:59.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:01.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:01.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:08.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:10.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:10.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:17.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:19.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:19.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:26.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:29.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:29.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:38.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:38.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:47.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:47.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:14:56.212 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:15:19.023 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-08-31T22:15:19.027 INFO:teuthology.orchestra.run.smithi187:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-08-31T22:15:27.957 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-08-31T22:15:27.957 INFO:teuthology.orchestra.run.smithi098:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-08-31T22:15:32.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:15:32.988 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-08-31T22:15:32.988 INFO:teuthology.orchestra.run.smithi026:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-08-31T22:15:33.343 INFO:teuthology.provision.fog.smithi026:Node is ready 2020-08-31T22:15:33.344 INFO:teuthology.orchestra.run.smithi026:> hostname 2020-08-31T22:15:33.398 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026.front.sepia.ceph.com 2020-08-31T22:15:33.399 INFO:teuthology.orchestra.run.smithi026:> grep smithi026.front.sepia.ceph.com /etc/hosts 2020-08-31T22:15:33.454 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26 smithi026.front.sepia.ceph.com smithi026 2020-08-31T22:15:33.454 INFO:teuthology.orchestra.run.smithi026:> sudo hostname smithi026 2020-08-31T22:15:33.573 INFO:teuthology.orchestra.run.smithi026:> sudo sed -i -e 's/smithi026.front.sepia.ceph.com/smithi026/g' /etc/hosts 2020-08-31T22:15:33.606 INFO:teuthology.orchestra.run.smithi026:> sudo sed -i -e 's/172.21.15.26/172.21.15.26/g' /etc/hosts 2020-08-31T22:15:33.679 INFO:teuthology.orchestra.run.smithi026:> sudo sed -i -e 's/smithi026.front.sepia.ceph.com/smithi026/g' /etc/hostname 2020-08-31T22:15:33.751 INFO:teuthology.orchestra.run.smithi026:> sudo hostname smithi026 2020-08-31T22:15:33.823 INFO:teuthology.provision.fog.smithi026:Deploy complete! 2020-08-31T22:15:33.824 DEBUG:teuthology.parallel:result is None 2020-08-31T22:15:39.903 INFO:teuthology.provision.fog.smithi187:Node is ready 2020-08-31T22:15:39.904 INFO:teuthology.orchestra.run.smithi187:> hostname 2020-08-31T22:15:39.928 INFO:teuthology.orchestra.run.smithi187.stdout:smithi187.front.sepia.ceph.com 2020-08-31T22:15:39.929 INFO:teuthology.orchestra.run.smithi187:> grep smithi187.front.sepia.ceph.com /etc/hosts 2020-08-31T22:15:39.985 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.15.187 smithi187.front.sepia.ceph.com smithi187 2020-08-31T22:15:39.985 INFO:teuthology.orchestra.run.smithi187:> sudo hostname smithi187 2020-08-31T22:15:40.127 INFO:teuthology.orchestra.run.smithi187:> sudo sed -i -e 's/smithi187.front.sepia.ceph.com/smithi187/g' /etc/hosts 2020-08-31T22:15:40.157 INFO:teuthology.orchestra.run.smithi187:> sudo sed -i -e 's/172.21.15.187/172.21.15.187/g' /etc/hosts 2020-08-31T22:15:40.226 INFO:teuthology.orchestra.run.smithi187:> sudo sed -i -e 's/smithi187.front.sepia.ceph.com/smithi187/g' /etc/hostname 2020-08-31T22:15:40.294 INFO:teuthology.orchestra.run.smithi187:> sudo hostname smithi187 2020-08-31T22:15:40.361 INFO:teuthology.provision.fog.smithi187:Deploy complete! 2020-08-31T22:15:40.362 DEBUG:teuthology.parallel:result is None 2020-08-31T22:15:48.728 INFO:teuthology.provision.fog.smithi098:Node is ready 2020-08-31T22:15:48.729 INFO:teuthology.orchestra.run.smithi098:> hostname 2020-08-31T22:15:48.748 INFO:teuthology.orchestra.run.smithi098.stdout:smithi098.front.sepia.ceph.com 2020-08-31T22:15:48.749 INFO:teuthology.orchestra.run.smithi098:> grep smithi098.front.sepia.ceph.com /etc/hosts 2020-08-31T22:15:48.805 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98 smithi098.front.sepia.ceph.com smithi098 2020-08-31T22:15:48.805 INFO:teuthology.orchestra.run.smithi098:> sudo hostname smithi098 2020-08-31T22:15:48.952 INFO:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hosts 2020-08-31T22:15:48.987 INFO:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/172.21.15.98/172.21.15.98/g' /etc/hosts 2020-08-31T22:15:49.056 INFO:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hostname 2020-08-31T22:15:49.124 INFO:teuthology.orchestra.run.smithi098:> sudo hostname smithi098 2020-08-31T22:15:49.192 INFO:teuthology.provision.fog.smithi098:Deploy complete! 2020-08-31T22:15:49.193 DEBUG:teuthology.parallel:result is None 2020-08-31T22:15:49.384 INFO:teuthology.lock.ops:Checking smithi026.front.sepia.ceph.com 2020-08-31T22:15:49.385 INFO:teuthology.lock.ops:New key found. Updating... 2020-08-31T22:15:49.403 INFO:teuthology.lock.ops:Checking smithi098.front.sepia.ceph.com 2020-08-31T22:15:49.404 INFO:teuthology.lock.ops:New key found. Updating... 2020-08-31T22:15:49.419 INFO:teuthology.lock.ops:Checking smithi187.front.sepia.ceph.com 2020-08-31T22:15:49.420 INFO:teuthology.lock.ops:New key found. Updating... 2020-08-31T22:15:49.436 INFO:teuthology.lock.ops:Updating [smithi026.front.sepia.ceph.com]: set os type and version on server 2020-08-31T22:15:49.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-08-31T22:15:49.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:15:49.513 INFO:teuthology.orchestra.run.smithi026:> true 2020-08-31T22:15:49.606 INFO:teuthology.orchestra.run.smithi026:> uname -m 2020-08-31T22:15:49.662 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2020-08-31T22:15:49.670 INFO:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2020-08-31T22:15:49.722 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Linux" 2020-08-31T22:15:49.723 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="8 (Core)" 2020-08-31T22:15:49.723 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2020-08-31T22:15:49.723 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2020-08-31T22:15:49.724 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="8" 2020-08-31T22:15:49.724 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el8" 2020-08-31T22:15:49.724 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-08-31T22:15:49.724 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2020-08-31T22:15:49.724 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-08-31T22:15:49.725 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://www.centos.org/" 2020-08-31T22:15:49.725 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-08-31T22:15:49.725 INFO:teuthology.orchestra.run.smithi026.stdout: 2020-08-31T22:15:49.725 INFO:teuthology.orchestra.run.smithi026.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-08-31T22:15:49.725 INFO:teuthology.orchestra.run.smithi026.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-08-31T22:15:49.726 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-08-31T22:15:49.726 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-08-31T22:15:49.726 INFO:teuthology.orchestra.run.smithi026.stdout: 2020-08-31T22:15:49.728 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2020-08-31T22:15:49.748 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: set os type and version on server 2020-08-31T22:15:49.748 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-08-31T22:15:49.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:15:49.817 INFO:teuthology.orchestra.run.smithi098:> true 2020-08-31T22:15:49.972 INFO:teuthology.orchestra.run.smithi098:> uname -m 2020-08-31T22:15:50.026 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2020-08-31T22:15:50.026 INFO:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2020-08-31T22:15:50.079 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Linux" 2020-08-31T22:15:50.079 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8 (Core)" 2020-08-31T22:15:50.080 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2020-08-31T22:15:50.080 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2020-08-31T22:15:50.080 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8" 2020-08-31T22:15:50.080 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8" 2020-08-31T22:15:50.080 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-08-31T22:15:50.081 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2020-08-31T22:15:50.081 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-08-31T22:15:50.081 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://www.centos.org/" 2020-08-31T22:15:50.082 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-08-31T22:15:50.082 INFO:teuthology.orchestra.run.smithi098.stdout: 2020-08-31T22:15:50.084 INFO:teuthology.orchestra.run.smithi098.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-08-31T22:15:50.084 INFO:teuthology.orchestra.run.smithi098.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-08-31T22:15:50.085 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-08-31T22:15:50.085 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-08-31T22:15:50.085 INFO:teuthology.orchestra.run.smithi098.stdout: 2020-08-31T22:15:50.086 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2020-08-31T22:15:50.102 INFO:teuthology.lock.ops:Updating [smithi187.front.sepia.ceph.com]: set os type and version on server 2020-08-31T22:15:50.103 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-08-31T22:15:50.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:15:50.175 INFO:teuthology.orchestra.run.smithi187:> true 2020-08-31T22:15:50.340 INFO:teuthology.orchestra.run.smithi187:> uname -m 2020-08-31T22:15:50.395 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2020-08-31T22:15:50.404 INFO:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2020-08-31T22:15:50.449 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="CentOS Linux" 2020-08-31T22:15:50.449 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="8 (Core)" 2020-08-31T22:15:50.449 INFO:teuthology.orchestra.run.smithi187.stdout:ID="centos" 2020-08-31T22:15:50.450 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="rhel fedora" 2020-08-31T22:15:50.450 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="8" 2020-08-31T22:15:50.450 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el8" 2020-08-31T22:15:50.450 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-08-31T22:15:50.450 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2020-08-31T22:15:50.451 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-08-31T22:15:50.451 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.centos.org/" 2020-08-31T22:15:50.451 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-08-31T22:15:50.451 INFO:teuthology.orchestra.run.smithi187.stdout: 2020-08-31T22:15:50.451 INFO:teuthology.orchestra.run.smithi187.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-08-31T22:15:50.452 INFO:teuthology.orchestra.run.smithi187.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-08-31T22:15:50.452 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-08-31T22:15:50.452 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-08-31T22:15:50.452 INFO:teuthology.orchestra.run.smithi187.stdout: 2020-08-31T22:15:50.453 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2020-08-31T22:15:50.469 INFO:teuthology.task.internal.lock_machines:3 smithi machines locked this try, 3/3 locked so far 2020-08-31T22:15:50.541 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi026.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAw4mgkWyraTTvsKriO+j2ibTJITnlJM5wS/n55Y1uiwEIiXzMY6ZKYXwBGF32Oed+ykW9+8cjBRv4s4/MQNcIvi/J7XBqMXd8cZ51YBBP97iNnQYm1ElT9VESAgf5EhtZCxI/YQXf+dbL7+wEHvd6D4+CSe1OIZhGFQ6ZelT8xxDMKxS9NbqPGrlaIl3prRY2qb/ecTSX6+V5g1Ye8eg5so/KZK3c2i1/7Sv1e/H1QdYa3bSevpPMvGlxvBTovqGzZYv7/EjVqkI7T7ceZKa3iqGOAREWJ5zeINkCucUzgLb6dtThb4jpGMMzkACrf4df7pZm3Vd3XNrvmCmPBEvDifRLlFxyWjJTEzszhXpwtfj4C7MHOwpuR4tV71Y7M2qbcM/S+dTKSFM8pPqreihGDvgPYR2H+PJ/OD+yFl3fJwuWvH+1EeW7ByWGOTLrNje856nuD73ziNiC3+f+CtwxzeKbbyag1wNtGcFBZl6Jg7QjooqXpar4WinkDqCYwGc= smithi098.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUFUYTU0Rv7OOvew/TfWwvWqlUeRYidMz/YQtc1OxE5QxOBy31FPKnU6YwHzqMvVZtwcwCrbaRpNMNk9mVDJRm9aLCSRShk5UPdIMMa+RmlA5eSTjUmvCa8r1jSCXYf6uo+y1+x51XNLFSdV2gjT+YgIQffO9JfU5m1Ynxmz1orPTuVWKfv8amupszlaqPXXlC4X0nSGwmHW5ORTo0fpztupd+V7meWw/V5dXMWoNU49UDm2bogH/cdSWd+lVBFEsfBxtoHPBNj1cbGlG8XHFULZ3UpiHF1IXhTsJ/y43rDca9W/ynfWBwhZr6afrvmbVAq6hNYcQVuDg0XepZZiaW88v+2ojk74IjBtDXEo01Ll4wJRquKepsFhWLC8LfX1hIoWkzbjnw+U0OUPKA6/gxrntQeyr11krR7yVJfM/vCigodKqFTp69keB7YveQ7QOB75dmNeO+6NtcJvpKDTsoiUlUrARb2R1OxLQaUjsXGzcKTU/x53ljEGnWL+jEigM= smithi187.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeukWt+hoBD/TxFrHztuYnij1aWLsyxi3Rrv+uhMQW/0mrglTZI0Sx0qnQKGJxqCRXEwyFfsnus09CY8LuwpL20ffTVpnmoWXz0VxTZzdzs/Bg3QP5xIQseSiZ2IEGISTDctCKeljKkEERVMf46HlUTXDTH2Xw479SxuEk13wNd8TyqSO9XFeTNIvusM8JEX7GAT+94L5+tldAIePdaCgeI0kcK/IdVNaA/r29DpXVvAHYmt4GS4ZyfbiBRnJ16w7fzF9TaVFWEgW+tdEKPOuZNRdiRgSuNZYV9X1XZiYdvAR8cJItPR10q/OvusEUsnTK9gdMb4MxczHmbFjmv/Y7LImqqlqKjBFOHxRAw5F3dp2SYxaMf3Lu5gWdoVp5enQVBcXdX0Fba7+8WiCefAbNJUjV+xm3/5CnT6UbXK1IwsYBJBAcuOtqSpqeWLYxYbFOaWIFrZuG59EmD39ojoYZv0DgDyTL3n2du+QUhNrB0LDeRZVAbGiERk5IwTv9MxU= 2020-08-31T22:15:50.542 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-08-31T22:15:50.610 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-08-31T22:15:50.618 INFO:teuthology.task.internal:Saving configuration 2020-08-31T22:15:50.638 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-08-31T22:15:50.647 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-08-31T22:15:50.669 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-08-31 22:11:02.762294', 'description': '/home/teuthworker/archive/pdonnell-2020-08-31_21:26:19-kcephfs-wip-pdonnell-testing-20200829.172552-distro-basic-smithi/5396255', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHoDGQaWGZLowQza4Gm9L6LkIvzU1uPA4/iEbtmU2eZD8SLbBouEZNp+FYOMdli5MtD7nKoZfFm6QQAKPPXlLY4=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi026.front.sepia.ceph.com'} 2020-08-31T22:15:50.688 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-08-31 22:11:02.762315', 'description': '/home/teuthworker/archive/pdonnell-2020-08-31_21:26:19-kcephfs-wip-pdonnell-testing-20200829.172552-distro-basic-smithi/5396255', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMSBK7Ci2Yj2LNpA6slKcuAH52lIq8NBqlb2xcB+U+DKjcRHADVkMqLnX5bgoOavvWqXapwswNH+U3XHuCtVxiA=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi098.front.sepia.ceph.com'} 2020-08-31T22:15:50.709 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-08-31 22:11:02.762331', 'description': '/home/teuthworker/archive/pdonnell-2020-08-31_21:26:19-kcephfs-wip-pdonnell-testing-20200829.172552-distro-basic-smithi/5396255', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK9s8OPTdUdnKfmm+txEi2VnYE2u6NFXgsKG8tbC6/ZjUE8w245gqdzS/zZQ/d3SERHP7x8GdOpHnKVdlpWIrYA=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi187.front.sepia.ceph.com'} 2020-08-31T22:15:50.710 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-08-31T22:15:50.722 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2020-08-31T22:15:50.723 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2020-08-31T22:15:50.724 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['client.0'] 2020-08-31T22:15:50.724 INFO:teuthology.run_tasks:Running task console_log... 2020-08-31T22:15:50.847 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fed57660d90>, signals=[15]) 2020-08-31T22:15:50.849 INFO:teuthology.run_tasks:Running task internal.connect... 2020-08-31T22:15:50.859 INFO:teuthology.task.internal:Opening connections... 2020-08-31T22:15:50.861 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2020-08-31T22:15:50.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:15:50.942 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi098.front.sepia.ceph.com 2020-08-31T22:15:50.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:15:51.028 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2020-08-31T22:15:51.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-31T22:15:51.107 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-08-31T22:15:51.120 INFO:teuthology.orchestra.run.smithi026:> uname -m 2020-08-31T22:15:51.137 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2020-08-31T22:15:51.138 INFO:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2020-08-31T22:15:51.192 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Linux" 2020-08-31T22:15:51.192 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="8 (Core)" 2020-08-31T22:15:51.193 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2020-08-31T22:15:51.193 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2020-08-31T22:15:51.193 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="8" 2020-08-31T22:15:51.193 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el8" 2020-08-31T22:15:51.194 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-08-31T22:15:51.194 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2020-08-31T22:15:51.194 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-08-31T22:15:51.195 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://www.centos.org/" 2020-08-31T22:15:51.195 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-08-31T22:15:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: 2020-08-31T22:15:51.195 INFO:teuthology.orchestra.run.smithi026.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-08-31T22:15:51.195 INFO:teuthology.orchestra.run.smithi026.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-08-31T22:15:51.198 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-08-31T22:15:51.198 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-08-31T22:15:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: 2020-08-31T22:15:51.199 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2020-08-31T22:15:51.223 INFO:teuthology.orchestra.run.smithi098:> uname -m 2020-08-31T22:15:51.238 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2020-08-31T22:15:51.239 INFO:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2020-08-31T22:15:51.293 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Linux" 2020-08-31T22:15:51.293 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8 (Core)" 2020-08-31T22:15:51.293 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2020-08-31T22:15:51.294 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2020-08-31T22:15:51.294 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8" 2020-08-31T22:15:51.294 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8" 2020-08-31T22:15:51.294 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-08-31T22:15:51.294 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2020-08-31T22:15:51.295 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-08-31T22:15:51.295 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://www.centos.org/" 2020-08-31T22:15:51.295 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-08-31T22:15:51.295 INFO:teuthology.orchestra.run.smithi098.stdout: 2020-08-31T22:15:51.295 INFO:teuthology.orchestra.run.smithi098.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-08-31T22:15:51.296 INFO:teuthology.orchestra.run.smithi098.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-08-31T22:15:51.296 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-08-31T22:15:51.296 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-08-31T22:15:51.296 INFO:teuthology.orchestra.run.smithi098.stdout: 2020-08-31T22:15:51.297 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2020-08-31T22:15:51.313 INFO:teuthology.orchestra.run.smithi187:> uname -m 2020-08-31T22:15:51.328 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2020-08-31T22:15:51.329 INFO:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2020-08-31T22:15:51.383 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="CentOS Linux" 2020-08-31T22:15:51.384 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="8 (Core)" 2020-08-31T22:15:51.384 INFO:teuthology.orchestra.run.smithi187.stdout:ID="centos" 2020-08-31T22:15:51.384 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="rhel fedora" 2020-08-31T22:15:51.384 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="8" 2020-08-31T22:15:51.385 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el8" 2020-08-31T22:15:51.385 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-08-31T22:15:51.385 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2020-08-31T22:15:51.385 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-08-31T22:15:51.386 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.centos.org/" 2020-08-31T22:15:51.386 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-08-31T22:15:51.387 INFO:teuthology.orchestra.run.smithi187.stdout: 2020-08-31T22:15:51.387 INFO:teuthology.orchestra.run.smithi187.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-08-31T22:15:51.387 INFO:teuthology.orchestra.run.smithi187.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-08-31T22:15:51.387 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-08-31T22:15:51.387 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-08-31T22:15:51.388 INFO:teuthology.orchestra.run.smithi187.stdout: 2020-08-31T22:15:51.388 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2020-08-31T22:15:51.406 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-08-31T22:15:51.505 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-08-31T22:15:51.516 INFO:teuthology.task.internal:Checking for old test directory... 2020-08-31T22:15:51.516 INFO:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2020-08-31T22:15:51.520 INFO:teuthology.orchestra.run.smithi098:> test '!' -e /home/ubuntu/cephtest 2020-08-31T22:15:51.523 INFO:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2020-08-31T22:15:51.537 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-08-31T22:15:51.547 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-08-31T22:15:51.548 INFO:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2020-08-31T22:15:51.576 INFO:teuthology.orchestra.run.smithi098:> test -z $(ls -A /var/lib/ceph) 2020-08-31T22:15:51.579 INFO:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2020-08-31T22:15:51.685 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-08-31T22:15:51.759 INFO:teuthology.run_tasks:Running task kernel... 2020-08-31T22:15:51.799 DEBUG:teuthology.task.kernel:normalized config {'client.0': {'branch': 'testing'}} 2020-08-31T22:15:51.800 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2020-08-31T22:15:51.806 INFO:teuthology.orchestra.run.smithi187:> sudo lsb_release -is 2020-08-31T22:15:51.859 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS 2020-08-31T22:15:51.861 DEBUG:teuthology.misc:System to be installed: CentOS 2020-08-31T22:15:51.862 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2020-08-31T22:15:52.105 DEBUG:teuthology.task.kernel:sha1 for client.0 is 7b4196f8b36937040ee59cf376f44604f885ef7e 2020-08-31T22:15:52.105 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "7b4196f8b36937040ee59cf376f44604f885ef7e"... 2020-08-31T22:15:52.106 INFO:teuthology.orchestra.run.smithi187:> uname -r 2020-08-31T22:15:52.130 INFO:teuthology.orchestra.run.smithi187.stdout:4.18.0-193.14.2.el8_2.x86_64 2020-08-31T22:15:52.130 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-193.14.2.el8_2.x86_64 vs 7b4196f8b36937040ee59cf376f44604f885ef7e 2020-08-31T22:15:52.131 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2020-08-31T22:15:52.132 INFO:teuthology.orchestra.run.smithi187:> sudo yum upgrade -y linux-firmware 2020-08-31T22:15:54.776 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-8 - AppStream 39 MB/s | 7.0 MB 00:00 2020-08-31T22:15:56.000 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-8 - Base 25 MB/s | 2.2 MB 00:00 2020-08-31T22:15:56.438 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-8 - Extras 74 kB/s | 5.5 kB 00:00 2020-08-31T22:15:56.894 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 19 MB/s | 7.9 MB 00:00 2020-08-31T22:15:58.489 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 3.9 MB/s | 20 kB 00:00 2020-08-31T22:15:59.621 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2020-08-31T22:15:59.622 INFO:teuthology.orchestra.run.smithi187.stdout:Nothing to do. 2020-08-31T22:15:59.622 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2020-08-31T22:15:59.770 INFO:teuthology.task.kernel:Downloading kernel 7b4196f8b36937040ee59cf376f44604f885ef7e on client.0... 2020-08-31T22:15:59.771 INFO:teuthology.orchestra.run.smithi187:> sudo lsb_release -is 2020-08-31T22:15:59.812 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS 2020-08-31T22:15:59.813 DEBUG:teuthology.misc:System to be installed: CentOS 2020-08-31T22:15:59.813 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b4196f8b36937040ee59cf376f44604f885ef7e 2020-08-31T22:16:00.004 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/7b4196f8b36937040ee59cf376f44604f885ef7e/centos/8/flavors/default/x86_64/ 2020-08-31T22:16:00.005 INFO:teuthology.orchestra.run.smithi187:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-5.9.0_rc2_ceph_g7b4196f8b369-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/7b4196f8b36937040ee59cf376f44604f885ef7e/centos/8/flavors/default/x86_64/ --input-file=- 2020-08-31T22:16:00.008 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi187.front.sepia.ceph.com to complete... 2020-08-31T22:16:39.618 INFO:teuthology.orchestra.run.smithi187.stderr:2020-08-31 22:16:39 URL:https://4.chacra.ceph.com/r/kernel/testing/7b4196f8b36937040ee59cf376f44604f885ef7e/centos/8/flavors/default/x86_64/kernel-5.9.0_rc2_ceph_g7b4196f8b369-1.x86_64.rpm [467932924/467932924] -> "/tmp/kernel.x86_64.rpm" [1] 2020-08-31T22:16:39.620 INFO:teuthology.orchestra.run.smithi187.stderr:FINISHED --2020-08-31 22:16:39-- 2020-08-31T22:16:39.620 INFO:teuthology.orchestra.run.smithi187.stderr:Total wall clock time: 40s 2020-08-31T22:16:39.620 INFO:teuthology.orchestra.run.smithi187.stderr:Downloaded: 1 files, 446M in 39s (11.3 MB/s) 2020-08-31T22:16:39.625 INFO:teuthology.task.kernel:Installing kernel 7b4196f8b36937040ee59cf376f44604f885ef7e on client.0... 2020-08-31T22:16:39.627 INFO:teuthology.orchestra.run.smithi187:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2020-08-31T22:16:41.550 INFO:teuthology.orchestra.run.smithi187.stdout:Verifying... ######################################## 2020-08-31T22:16:42.022 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing... ######################################## 2020-08-31T22:16:42.084 INFO:teuthology.orchestra.run.smithi187.stdout:Updating / installing... 2020-08-31T22:17:16.635 INFO:teuthology.orchestra.run.smithi187.stdout:kernel-5.9.0_rc2_ceph_g7b4196f8b369-1 ######################################## 2020-08-31T22:17:26.488 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2020-08-31T22:17:26.490 DEBUG:teuthology.task.console_log:Killing console logger for smithi026 2020-08-31T22:17:26.491 DEBUG:teuthology.task.console_log:Killing console logger for smithi098 2020-08-31T22:17:26.492 DEBUG:teuthology.task.console_log:Killing console logger for smithi187 2020-08-31T22:17:26.492 DEBUG:teuthology.task.console_log:Killing console logger for smithi026 2020-08-31T22:17:26.493 DEBUG:teuthology.task.console_log:Killing console logger for smithi098 2020-08-31T22:17:26.493 DEBUG:teuthology.task.console_log:Killing console logger for smithi187 2020-08-31T22:17:26.493 DEBUG:teuthology.exit:Finished running handlers