2021-02-10T22:13:07.318 INFO:root:teuthology version: 1.1.0-14fad317 2021-02-10T22:13:07.319 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_nojha@teuthology --archive /home/teuthworker/archive/nojha-2021-02-10_22:05:23-rados:cephadm-master-distro-basic-smithi/5873883 --name nojha-2021-02-10_22:05:23-rados:cephadm-master-distro-basic-smithi --description rados:cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_orch_cli} --verbose -- /home/teuthworker/archive/nojha-2021-02-10_22:05:23-rados:cephadm-master-distro-basic-smithi/5873883/orig.config.yaml 2021-02-10T22:13:07.347 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-02-10T22:13:07.414 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/nojha-2021-02-10_22:05:23-rados:cephadm-master-distro-basic-smithi/5873883 branch: master description: rados:cephadm/workunits/{distro/centos_latest mon_election/connectivity task/test_orch_cli} email: null first_in_suite: false job_id: '5873883' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: nojha-2021-02-10_22:05:23-rados:cephadm-master-distro-basic-smithi nuke-on-error: true os_type: centos os_version: '8.2' overrides: admin_socket: branch: master ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: b4d9cc45d6ff1ea5382954dece424128b478d6f7 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: b4d9cc45d6ff1ea5382954dece424128b478d6f7 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: master sha1: b4d9cc45d6ff1ea5382954dece424128b478d6f7 owner: scheduled_nojha@teuthology priority: 21 repo: https://github.com/ceph/ceph.git roles: - - host.a - osd.0 - osd.1 - osd.2 - mon.a - mgr.a - client.0 sha1: b4d9cc45d6ff1ea5382954dece424128b478d6f7 sleep_before_teardown: 0 suite: rados:cephadm suite_branch: master suite_path: /home/teuthworker/src/github.com_ceph_ceph_b4d9cc45d6ff1ea5382954dece424128b478d6f7/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: b4d9cc45d6ff1ea5382954dece424128b478d6f7 targets: smithi016.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTAhadbt6hZ2i4kojlDwwbjExKHdEfn24FMiK7zPhSZttKG4ZtiCSnA254LFG2CYZOW/FGGqRb7TOjxLShRXk2LMkN/OKq4LZBp9Jp15c8kAsM5D24m8CD+aUvDtP6v5fqadYzof6g46vx0IjuZ+P4yUoNXPbJebgps8PBRdOXiH2Ndk0qzzuna9WzdzP8zzaueKryYxK3Jss0bPt9BL3RUoZpXWWIknEUj9p06xs6UAFvr9mKfuSLZviCcXLY2B1oRoM/vD6gram9Ll0+kLj5llQ5b3txJgzWuyCfioQ85X1MHXzAaIuA3/AwewmogcEVk7BCX/OeVbC9Zqh4PgLMm8D+png9om0984Fdi3tjvjEurOGVBCx71vKABeaRNJueQh1ZQj3hP3j7OQUrQzN+ZJoU+bmwHn/ru13yDwftiK5y2CdAhve/CzM9EAla6TCUuCY/w1N/UFzbsZuLs+4Oj3P9CaonCoyuGxVJsOCTBKQSp/2TM3MAwAOWUtmrEDE= tasks: - install: null - cephadm: null - cephadm.shell: host.a: - ceph orch apply mds a - cephfs_test_runner: modules: - tasks.cephfs.test_nfs - tasks.cephadm_cases.test_cli teuthology_branch: master teuthology_sha1: 14fad31725ce3e5fc27890f2a8ef1f21c838d296 timestamp: 2021-02-10_22:05:23 tube: smithi user: nojha verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.29888 2021-02-10T22:13:07.414 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_b4d9cc45d6ff1ea5382954dece424128b478d6f7/qa; will attempt to use it 2021-02-10T22:13:07.426 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_b4d9cc45d6ff1ea5382954dece424128b478d6f7/qa/tasks 2021-02-10T22:13:07.426 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-02-10T22:13:07.428 INFO:teuthology.task.internal:Checking packages... 2021-02-10T22:13:07.479 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash 'b4d9cc45d6ff1ea5382954dece424128b478d6f7' 2021-02-10T22:13:07.479 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-02-10T22:13:07.479 INFO:teuthology.packaging:ref: None 2021-02-10T22:13:07.479 INFO:teuthology.packaging:tag: None 2021-02-10T22:13:07.480 INFO:teuthology.packaging:branch: master 2021-02-10T22:13:07.480 INFO:teuthology.packaging:sha1: b4d9cc45d6ff1ea5382954dece424128b478d6f7 2021-02-10T22:13:07.480 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=master 2021-02-10T22:13:08.412 INFO:teuthology.task.internal:Found packages for ceph version 17.0.0-703.gb4d9cc45 2021-02-10T22:13:08.412 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-02-10T22:13:08.418 INFO:teuthology.task.internal:no buildpackages task found 2021-02-10T22:13:08.419 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-02-10T22:13:08.953 INFO:teuthology.task.internal:Saving configuration 2021-02-10T22:13:08.963 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-02-10T22:13:09.274 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-10T22:13:09.295 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-02-10 22:08:24.456201', 'description': '/home/teuthworker/archive/nojha-2021-02-10_22:05:23-rados:cephadm-master-distro-basic-smithi/5873883', 'locked_by': 'scheduled_nojha@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBArni4QKK/ZCt3BFPpD1AstzOKOE03Zb80cVkxmICwohRzBe0K9EZqVR8Q6UsikS4YH0toXT/rfEWcHBoRQOMbI=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi016.front.sepia.ceph.com'} 2021-02-10T22:13:09.296 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2021-02-10T22:13:09.489 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0'] 2021-02-10T22:13:09.489 INFO:teuthology.run_tasks:Running task console_log... 2021-02-10T22:13:09.858 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3e527daf28>, signals=[15]) 2021-02-10T22:13:09.858 INFO:teuthology.run_tasks:Running task internal.connect... 2021-02-10T22:13:11.065 INFO:teuthology.task.internal:Opening connections... 2021-02-10T22:13:11.066 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2021-02-10T22:13:11.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-02-10T22:13:11.145 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2021-02-10T22:13:11.517 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2021-02-10T22:13:11.535 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2021-02-10T22:13:11.536 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2021-02-10T22:13:11.592 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Linux" 2021-02-10T22:13:11.592 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8 (Core)" 2021-02-10T22:13:11.592 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2021-02-10T22:13:11.592 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2021-02-10T22:13:11.592 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8" 2021-02-10T22:13:11.592 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2021-02-10T22:13:11.593 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-02-10T22:13:11.593 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2021-02-10T22:13:11.593 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-02-10T22:13:11.593 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.centos.org/" 2021-02-10T22:13:11.593 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-02-10T22:13:11.593 INFO:teuthology.orchestra.run.smithi016.stdout: 2021-02-10T22:13:11.593 INFO:teuthology.orchestra.run.smithi016.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-02-10T22:13:11.593 INFO:teuthology.orchestra.run.smithi016.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-02-10T22:13:11.593 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-02-10T22:13:11.593 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-02-10T22:13:11.593 INFO:teuthology.orchestra.run.smithi016.stdout: 2021-02-10T22:13:11.594 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2021-02-10T22:13:11.620 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2021-02-10T22:13:12.074 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2021-02-10T22:13:12.159 INFO:teuthology.task.internal:Checking for old test directory... 2021-02-10T22:13:12.160 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2021-02-10T22:13:12.176 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2021-02-10T22:13:13.262 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2021-02-10T22:13:13.262 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2021-02-10T22:13:13.290 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2021-02-10T22:13:13.406 INFO:teuthology.run_tasks:Running task kernel... 2021-02-10T22:13:13.618 DEBUG:teuthology.task.kernel:normalized config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.a': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}} 2021-02-10T22:13:13.619 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2021-02-10T22:13:13.619 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2021-02-10T22:13:13.635 INFO:teuthology.orchestra.run.smithi016.stdout:4.18.0-193.28.1.el8_2.x86_64 2021-02-10T22:13:13.636 INFO:teuthology.task.kernel:Running kernel on smithi016: 4.18.0-193.28.1.el8_2.x86_64 2021-02-10T22:13:13.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2021-02-10T22:13:16.970 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-8 - AppStream 13 MB/s | 6.3 MB 00:00 2021-02-10T22:13:18.235 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-8 - Base 12 MB/s | 2.3 MB 00:00 2021-02-10T22:13:19.670 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-8 - Extras 8.4 kB/s | 8.6 kB 00:01 2021-02-10T22:13:20.089 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 25 MB/s | 8.8 MB 00:00 2021-02-10T22:13:22.040 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2021-02-10T22:13:23.096 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-4.18.0-193.el8.x86_64 is already installed. 2021-02-10T22:13:23.096 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-4.18.0-193.28.1.el8_2.x86_64 is already installed. 2021-02-10T22:13:23.292 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2021-02-10T22:13:23.303 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2021-02-10T22:13:23.304 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2021-02-10T22:13:23.304 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2021-02-10T22:13:23.304 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2021-02-10T22:13:23.304 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 4.18.0-240.10.1.el8_3 CentOS-Base 4.3 M 2021-02-10T22:13:23.304 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2021-02-10T22:13:23.304 INFO:teuthology.orchestra.run.smithi016.stdout: linux-firmware noarch 20200619-101.git3890db36.el8_3 CentOS-Base 101 M 2021-02-10T22:13:23.304 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2021-02-10T22:13:23.304 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core x86_64 4.18.0-240.10.1.el8_3 CentOS-Base 30 M 2021-02-10T22:13:23.304 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules x86_64 4.18.0-240.10.1.el8_3 CentOS-Base 26 M 2021-02-10T22:13:23.304 INFO:teuthology.orchestra.run.smithi016.stdout: 2021-02-10T22:13:23.304 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2021-02-10T22:13:23.304 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2021-02-10T22:13:23.304 INFO:teuthology.orchestra.run.smithi016.stdout:Install 3 Packages 2021-02-10T22:13:23.305 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 1 Package 2021-02-10T22:13:23.305 INFO:teuthology.orchestra.run.smithi016.stdout: 2021-02-10T22:13:23.305 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 162 M 2021-02-10T22:13:23.305 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2021-02-10T22:13:24.518 INFO:teuthology.orchestra.run.smithi016.stdout:(1/4): kernel-4.18.0-240.10.1.el8_3.x86_64.rpm 3.6 MB/s | 4.3 MB 00:01 2021-02-10T22:13:26.511 INFO:teuthology.orchestra.run.smithi016.stdout:(2/4): kernel-core-4.18.0-240.10.1.el8_3.x86_64 9.5 MB/s | 30 MB 00:03 2021-02-10T22:13:27.134 INFO:teuthology.orchestra.run.smithi016.stdout:(3/4): kernel-modules-4.18.0-240.10.1.el8_3.x86 6.7 MB/s | 26 MB 00:03 2021-02-10T22:13:32.284 INFO:teuthology.orchestra.run.smithi016.stdout:(4/4): linux-firmware-20200619-101.git3890db36. 13 MB/s | 101 MB 00:07 2021-02-10T22:13:32.287 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2021-02-10T22:13:32.287 INFO:teuthology.orchestra.run.smithi016.stdout:Total 18 MB/s | 162 MB 00:08 2021-02-10T22:13:32.414 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2021-02-10T22:13:32.699 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2021-02-10T22:13:32.700 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2021-02-10T22:13:34.399 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2021-02-10T22:13:34.400 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2021-02-10T22:13:44.900 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2021-02-10T22:13:46.747 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : linux-firmware-20200619-101.git3890db36.el8_3.noarch 1/5 2021-02-10T22:13:47.274 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-core-4.18.0-240.10.1.el8_3.x86_64 2/5 2021-02-10T22:13:50.181 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-4.18.0-240.10.1.el8_3.x86_64 2/5 2021-02-10T22:13:50.507 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-4.18.0-240.10.1.el8_3.x86_64 3/5 2021-02-10T22:13:53.604 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-4.18.0-240.10.1.el8_3.x86_64 3/5 2021-02-10T22:13:54.026 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-4.18.0-240.10.1.el8_3.x86_64 4/5 2021-02-10T22:13:54.252 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : linux-firmware-20191202-97.gite8a0f4c9.el8.noarch 5/5 2021-02-10T22:13:59.804 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2021-02-10T22:13:59.805 DEBUG:teuthology.task.console_log:Killing console logger for smithi016 2021-02-10T22:13:59.805 DEBUG:teuthology.exit:Finished running handlers