2020-06-12T02:32:51.572 INFO:root:teuthology version: 1.0.0-6c966fe 2020-06-12T02:32:51.574 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/kchai-2020-06-11_10:13:29-rados-octopus-distro-basic-smithi/5138065 --description rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-partial-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} --block --name kchai-2020-06-11_10:13:29-rados-octopus-distro-basic-smithi --lock --verbose --owner scheduled_kchai@teuthology -- /tmp/teuthology-worker._2mk2w9s.tmp 2020-06-12T02:32:51.613 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-06-12T02:32:51.688 INFO:teuthology.run:Config: arch: x86_64 archive_path: /home/teuthworker/archive/kchai-2020-06-11_10:13:29-rados-octopus-distro-basic-smithi/5138065 branch: octopus description: rados/thrash-erasure-code-isa/{arch/x86_64.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml rados.yaml recovery-overrides/{more-partial-recovery.yaml} supported-random-distro$/{centos_latest.yaml} thrashers/mapgap.yaml thrashosds-health.yaml workloads/ec-rados-plugin=isa-k=2-m=1.yaml} email: tchaikov@gmail.com first_in_suite: false job_id: '5138065' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: kchai-2020-06-11_10:13:29-rados-octopus-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: centos os_version: '8.1' overrides: admin_socket: branch: octopus ceph: conf: global: mon client directed command retry: 5 ms inject socket failures: 5000 osd_object_clean_region_max_num_intervals: 1000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd heartbeat use min delay socket: true osd map cache size: 1 osd max backfills: 6 osd max markdown count: 1000 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub during recovery: false osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - osd_map_cache_size - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request sha1: 91d24f498556f86f6d1318d2efd5210c35b36762 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: sha1: 91d24f498556f86f6d1318d2efd5210c35b36762 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: octopus-py3 sha1: 1e0d5d60071d5554f87a2be72ea0bcccb272aab3 owner: scheduled_kchai@teuthology priority: 90 repo: https://github.com/ceph/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: 91d24f498556f86f6d1318d2efd5210c35b36762 sleep_before_teardown: 0 suite: rados suite_branch: octopus-py3 suite_path: /home/teuthworker/src/github.com_tchaikov_ceph_octopus-py3/qa suite_relpath: qa suite_repo: https://github.com/tchaikov/ceph.git suite_sha1: 1e0d5d60071d5554f87a2be72ea0bcccb272aab3 tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: chance_pgnum_grow: 0.25 chance_pgnum_shrink: 0.25 chance_pgpnum_fix: 0.25 chance_test_map_discontinuity: 2 timeout: 1800 - rados: clients: - client.0 ec_pool: true erasure_code_profile: crush-failure-domain: osd k: 2 m: 1 name: isaprofile plugin: isa technique: reed_sol_van min_size: 2 objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 4000 write_append_excl: false teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.20699 2020-06-12T02:32:51.688 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_tchaikov_ceph_octopus-py3/qa; will attempt to use it 2020-06-12T02:32:51.690 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_tchaikov_ceph_octopus-py3/qa/tasks 2020-06-12T02:32:51.691 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-06-12T02:32:51.692 INFO:teuthology.task.internal:Checking packages... 2020-06-12T02:32:51.719 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash '91d24f498556f86f6d1318d2efd5210c35b36762' 2020-06-12T02:32:51.720 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-06-12T02:32:51.720 INFO:teuthology.packaging:ref: None 2020-06-12T02:32:51.721 INFO:teuthology.packaging:tag: None 2020-06-12T02:32:51.721 INFO:teuthology.packaging:branch: octopus 2020-06-12T02:32:51.721 INFO:teuthology.packaging:sha1: 91d24f498556f86f6d1318d2efd5210c35b36762 2020-06-12T02:32:51.722 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2020-06-12T02:32:51.997 INFO:teuthology.task.internal:Found packages for ceph version 15.2.3-298.g7488806bf1 2020-06-12T02:32:51.997 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-06-12T02:32:52.008 INFO:teuthology.task.internal:no buildpackages task found 2020-06-12T02:32:52.008 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-06-12T02:32:52.023 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-06-12T02:32:52.024 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-06-12T02:32:52.080 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-06-12T02:33:02.099 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-06-12T02:33:12.117 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-06-12T02:33:22.138 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-06-12T02:33:32.159 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-06-12T02:33:42.183 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-06-12T02:33:52.202 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-06-12T02:34:02.219 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-06-12T02:34:12.241 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_kchai@teuthology', 'count': 2, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/kchai-2020-06-11_10:13:29-rados-octopus-distro-basic-smithi/5138065', 'arch': 'x86_64'} 2020-06-12T02:34:12.269 DEBUG:teuthology.lock.ops:locked ubuntu@smithi014.front.sepia.ceph.com, ubuntu@smithi089.front.sepia.ceph.com 2020-06-12T02:34:12.344 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdd584c1ae8>, signals=[15]) 2020-06-12T02:34:12.542 INFO:teuthology.provision.fog.smithi014:Scheduling deploy of centos 8.1 2020-06-12T02:34:12.564 INFO:teuthology.provision.fog.smithi089:Scheduling deploy of centos 8.1 2020-06-12T02:34:12.896 INFO:teuthology.orchestra.console:Power off smithi014 2020-06-12T02:34:12.898 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi014.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-06-12T02:34:12.929 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi014.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-06-12T02:34:12.958 INFO:teuthology.orchestra.console:Power off smithi089 2020-06-12T02:34:12.959 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-06-12T02:34:12.985 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-06-12T02:34:16.975 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi014.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-06-12T02:34:17.018 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-06-12T02:34:21.094 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi014.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-06-12T02:34:21.141 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-06-12T02:34:21.309 INFO:teuthology.orchestra.console:Power off for smithi014 completed 2020-06-12T02:34:21.414 INFO:teuthology.orchestra.console:Power on smithi014 2020-06-12T02:34:21.414 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi014.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-06-12T02:34:21.438 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi014.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-06-12T02:34:25.256 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-06-12T02:34:25.467 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi014.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-06-12T02:34:25.481 INFO:teuthology.orchestra.console:Power off for smithi089 completed 2020-06-12T02:34:25.583 INFO:teuthology.orchestra.console:Power on smithi089 2020-06-12T02:34:25.583 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-06-12T02:34:25.609 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-06-12T02:34:29.583 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi014.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-06-12T02:34:29.635 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-06-12T02:34:29.862 INFO:teuthology.orchestra.console:Power on for smithi089 completed 2020-06-12T02:34:29.866 INFO:teuthology.orchestra.console:Power on for smithi014 completed 2020-06-12T02:34:29.964 INFO:teuthology.provision.fog.smithi089:Waiting for deploy to finish 2020-06-12T02:34:29.970 INFO:teuthology.provision.fog.smithi014:Waiting for deploy to finish 2020-06-12T02:36:46.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi014.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:36:55.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi014.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:37:01.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:37:04.530 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi014.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:37:10.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:37:13.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi014.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:37:19.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:37:22.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi014.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:37:28.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:37:31.701 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi014.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:37:37.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:37:46.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:37:55.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:38:20.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:38:29.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:38:29.571 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-06-12T02:38:29.574 INFO:teuthology.orchestra.run.smithi089:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-06-12T02:38:37.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi014.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:38:38.078 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-06-12T02:38:38.079 INFO:teuthology.orchestra.run.smithi014:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-06-12T02:38:38.447 INFO:teuthology.provision.fog.smithi014:Node is ready 2020-06-12T02:38:38.448 INFO:teuthology.orchestra.run.smithi014:> hostname 2020-06-12T02:38:38.463 INFO:teuthology.orchestra.run.smithi014.stdout:smithi014.front.sepia.ceph.com 2020-06-12T02:38:38.464 INFO:teuthology.orchestra.run.smithi014:> grep smithi014.front.sepia.ceph.com /etc/hosts 2020-06-12T02:38:38.519 INFO:teuthology.orchestra.run.smithi014.stdout:172.21.15.14 smithi014.front.sepia.ceph.com smithi014 2020-06-12T02:38:38.520 INFO:teuthology.orchestra.run.smithi014:> sudo hostname smithi014 2020-06-12T02:38:38.633 INFO:teuthology.orchestra.run.smithi014:> sudo sed -i -e 's/smithi014.front.sepia.ceph.com/smithi014/g' /etc/hosts 2020-06-12T02:38:38.666 INFO:teuthology.orchestra.run.smithi014:> sudo sed -i -e 's/172.21.15.14/172.21.15.14/g' /etc/hosts 2020-06-12T02:38:38.739 INFO:teuthology.orchestra.run.smithi014:> sudo sed -i -e 's/smithi014.front.sepia.ceph.com/smithi014/g' /etc/hostname 2020-06-12T02:38:38.815 INFO:teuthology.orchestra.run.smithi014:> sudo hostname smithi014 2020-06-12T02:38:38.888 INFO:teuthology.provision.fog.smithi014:Deploy complete! 2020-06-12T02:38:38.891 DEBUG:teuthology.parallel:result is None 2020-06-12T02:38:49.586 INFO:teuthology.provision.fog.smithi089:Node is ready 2020-06-12T02:38:49.587 INFO:teuthology.orchestra.run.smithi089:> hostname 2020-06-12T02:38:49.606 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089.front.sepia.ceph.com 2020-06-12T02:38:49.607 INFO:teuthology.orchestra.run.smithi089:> grep smithi089.front.sepia.ceph.com /etc/hosts 2020-06-12T02:38:49.663 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89 smithi089.front.sepia.ceph.com smithi089 2020-06-12T02:38:49.664 INFO:teuthology.orchestra.run.smithi089:> sudo hostname smithi089 2020-06-12T02:38:49.816 INFO:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/smithi089.front.sepia.ceph.com/smithi089/g' /etc/hosts 2020-06-12T02:38:49.847 INFO:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/172.21.15.89/172.21.15.89/g' /etc/hosts 2020-06-12T02:38:49.916 INFO:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/smithi089.front.sepia.ceph.com/smithi089/g' /etc/hostname 2020-06-12T02:38:49.984 INFO:teuthology.orchestra.run.smithi089:> sudo hostname smithi089 2020-06-12T02:38:50.051 INFO:teuthology.provision.fog.smithi089:Deploy complete! 2020-06-12T02:38:50.052 DEBUG:teuthology.parallel:result is None 2020-06-12T02:38:50.186 INFO:teuthology.lock.ops:Checking smithi014.front.sepia.ceph.com 2020-06-12T02:38:50.187 INFO:teuthology.lock.ops:New key found. Updating... 2020-06-12T02:38:50.228 INFO:teuthology.lock.ops:Checking smithi089.front.sepia.ceph.com 2020-06-12T02:38:50.229 INFO:teuthology.lock.ops:New key found. Updating... 2020-06-12T02:38:50.249 INFO:teuthology.lock.ops:Updating [smithi014.front.sepia.ceph.com]: set os type and version on server 2020-06-12T02:38:50.250 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-06-12T02:38:50.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi014.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:38:50.324 INFO:teuthology.orchestra.run.smithi014:> true 2020-06-12T02:38:50.511 INFO:teuthology.orchestra.run.smithi014:> uname -m 2020-06-12T02:38:50.566 INFO:teuthology.orchestra.run.smithi014.stdout:x86_64 2020-06-12T02:38:50.566 INFO:teuthology.orchestra.run.smithi014:> cat /etc/os-release 2020-06-12T02:38:50.621 INFO:teuthology.orchestra.run.smithi014.stdout:NAME="CentOS Linux" 2020-06-12T02:38:50.621 INFO:teuthology.orchestra.run.smithi014.stdout:VERSION="8 (Core)" 2020-06-12T02:38:50.621 INFO:teuthology.orchestra.run.smithi014.stdout:ID="centos" 2020-06-12T02:38:50.622 INFO:teuthology.orchestra.run.smithi014.stdout:ID_LIKE="rhel fedora" 2020-06-12T02:38:50.622 INFO:teuthology.orchestra.run.smithi014.stdout:VERSION_ID="8" 2020-06-12T02:38:50.622 INFO:teuthology.orchestra.run.smithi014.stdout:PLATFORM_ID="platform:el8" 2020-06-12T02:38:50.622 INFO:teuthology.orchestra.run.smithi014.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-06-12T02:38:50.622 INFO:teuthology.orchestra.run.smithi014.stdout:ANSI_COLOR="0;31" 2020-06-12T02:38:50.623 INFO:teuthology.orchestra.run.smithi014.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-06-12T02:38:50.623 INFO:teuthology.orchestra.run.smithi014.stdout:HOME_URL="https://www.centos.org/" 2020-06-12T02:38:50.623 INFO:teuthology.orchestra.run.smithi014.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-06-12T02:38:50.623 INFO:teuthology.orchestra.run.smithi014.stdout: 2020-06-12T02:38:50.623 INFO:teuthology.orchestra.run.smithi014.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-06-12T02:38:50.623 INFO:teuthology.orchestra.run.smithi014.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-06-12T02:38:50.624 INFO:teuthology.orchestra.run.smithi014.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-06-12T02:38:50.624 INFO:teuthology.orchestra.run.smithi014.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-06-12T02:38:50.624 INFO:teuthology.orchestra.run.smithi014.stdout: 2020-06-12T02:38:50.625 INFO:teuthology.lock.ops:Updating smithi014.front.sepia.ceph.com on lock server 2020-06-12T02:38:50.651 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: set os type and version on server 2020-06-12T02:38:50.652 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-06-12T02:38:50.652 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:38:50.721 INFO:teuthology.orchestra.run.smithi089:> true 2020-06-12T02:38:50.879 INFO:teuthology.orchestra.run.smithi089:> uname -m 2020-06-12T02:38:50.933 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2020-06-12T02:38:50.934 INFO:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2020-06-12T02:38:50.987 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Linux" 2020-06-12T02:38:50.988 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8 (Core)" 2020-06-12T02:38:50.988 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2020-06-12T02:38:50.988 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2020-06-12T02:38:50.988 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8" 2020-06-12T02:38:50.989 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8" 2020-06-12T02:38:50.989 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-06-12T02:38:50.989 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2020-06-12T02:38:50.989 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-06-12T02:38:50.991 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://www.centos.org/" 2020-06-12T02:38:50.991 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-06-12T02:38:50.992 INFO:teuthology.orchestra.run.smithi089.stdout: 2020-06-12T02:38:50.993 INFO:teuthology.orchestra.run.smithi089.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-06-12T02:38:50.993 INFO:teuthology.orchestra.run.smithi089.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-06-12T02:38:50.994 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-06-12T02:38:50.994 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-06-12T02:38:50.994 INFO:teuthology.orchestra.run.smithi089.stdout: 2020-06-12T02:38:50.995 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2020-06-12T02:38:51.016 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2020-06-12T02:38:51.052 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi014.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvTE1WVdNjenQHJ+OvMzeWII7rM0AoysX4pGtOWqEhuCEGtWXFWE1vacARi5HheTKU6ZdmEaeddYtZBqMnkqjLqMZ2ylwHTUw0wX3mgDjR7GqGYJBbp8ZZBFhgH1XNya/GFaz/GPWS0fSyWpGXxxnsb6Q57U0c+ay638hdXt17pA0DFN60Xe5jS/iwwg+geI9Foe7QL1ANAyVtivZauv8n6EnAE3mEID6ijZn6WUbcBuUdS7KDk8jezFysvgvpqzOm1jZntQ3oebo70Za9P0ENiMoceJeZJn3PjFsHPuJIjXYbF9JJgCKpeV0BTOe8sxdGWohrGi1QEEWLHmcugb5llJHemHwisP3KTPKFatxBIKv41P8yZ+LeBiZMCGTfhgRY/jHgUqJXd71T6+aOBSFLNROxS2iMK1ABq9xBVEIdJXFmFx9d8MUXZPrCh5PqJhTY2ImzxVoUeGctRyXGzR0+j+ea3pIqjwwpiaNJMp2itS5Xj7TdR69UDdGo9Uk4yM0= smithi089.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCudxInpYbztPp23SgfJPpiungg+yRqOpUCK8paFtKGWC0iMcYtw+zvGjZmNI3jAe9AJAu9dI4Fm8pCn84OZHCHjcRpY24bICofJER+cPflKOVTnDbzR3RH/3Eh/I8UpjRF6VYui1znGyUCuad34BA/5s5G6k96SEgED6Y5KL911+MK14CbNyJdoWz9zmfJ3sPNWRqtWpvbdCvLhLhsvQhj8H4Hav1ZlApoJSvlGHWEKHPYW66KGLCmHyNjgPa20KBtlrl5alHk27IBugRR4T5oWMDks6F0XG3l/aFFJHUJ2iIJurdyBbeC2uJ8OR1Pfhbu21RiOZNULhaE69pxlqB2g869tT39lz4XNALS69IhY4GzDA+xyLPL5ulzzvs96kAHkTYydZ4QCvyb4ZBBbszq9SFS0vnK3SwjYMm2Ul7lfMt9vVRO9eOkJHN1MnfkcaQFnc1/TMFrRMv51GSUfRM6UOqIUDwwEZDUsFyFJN48uwgMt4tY+/1ZxOCLIP4AyWU= 2020-06-12T02:38:51.054 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-06-12T02:38:51.110 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-06-12T02:38:51.121 INFO:teuthology.task.internal:Saving configuration 2020-06-12T02:38:51.145 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-06-12T02:38:51.163 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-06-12T02:38:51.183 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-06-12 02:34:12.249766', 'description': '/home/teuthworker/archive/kchai-2020-06-11_10:13:29-rados-octopus-distro-basic-smithi/5138065', 'locked_by': 'scheduled_kchai@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBdkFctzqto5bxJp8igBYDbnMpG5uGaigI09J70TFdCuiIKZ37T5hmifpZK4Q/VTtX4tJ1+EJZ4Wtp0ZRQ1GBfs=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi014.front.sepia.ceph.com'} 2020-06-12T02:38:51.199 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-06-12 02:34:12.249786', 'description': '/home/teuthworker/archive/kchai-2020-06-11_10:13:29-rados-octopus-distro-basic-smithi/5138065', 'locked_by': 'scheduled_kchai@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgJjMaKBkp+DSFQzPF9HPPas+2+rw9Yh2sMUUbTdEnFMrvvZS6w7mg5UpnovbRM2f8xCXbOPj3pHD8WxzSXShw=', 'os_version': '8', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'centos', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi089.front.sepia.ceph.com'} 2020-06-12T02:38:51.200 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-06-12T02:38:51.210 INFO:teuthology.task.internal:roles: ubuntu@smithi014.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2020-06-12T02:38:51.211 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2020-06-12T02:38:51.211 INFO:teuthology.run_tasks:Running task console_log... 2020-06-12T02:38:51.318 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdd5943dd08>, signals=[15]) 2020-06-12T02:38:51.319 INFO:teuthology.run_tasks:Running task internal.connect... 2020-06-12T02:38:51.331 INFO:teuthology.task.internal:Opening connections... 2020-06-12T02:38:51.331 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi014.front.sepia.ceph.com 2020-06-12T02:38:51.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi014.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:38:51.437 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2020-06-12T02:38:51.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-06-12T02:38:51.510 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-06-12T02:38:51.523 INFO:teuthology.orchestra.run.smithi014:> uname -m 2020-06-12T02:38:51.583 INFO:teuthology.orchestra.run.smithi014.stdout:x86_64 2020-06-12T02:38:51.583 INFO:teuthology.orchestra.run.smithi014:> cat /etc/os-release 2020-06-12T02:38:51.637 INFO:teuthology.orchestra.run.smithi014.stdout:NAME="CentOS Linux" 2020-06-12T02:38:51.637 INFO:teuthology.orchestra.run.smithi014.stdout:VERSION="8 (Core)" 2020-06-12T02:38:51.638 INFO:teuthology.orchestra.run.smithi014.stdout:ID="centos" 2020-06-12T02:38:51.638 INFO:teuthology.orchestra.run.smithi014.stdout:ID_LIKE="rhel fedora" 2020-06-12T02:38:51.638 INFO:teuthology.orchestra.run.smithi014.stdout:VERSION_ID="8" 2020-06-12T02:38:51.638 INFO:teuthology.orchestra.run.smithi014.stdout:PLATFORM_ID="platform:el8" 2020-06-12T02:38:51.639 INFO:teuthology.orchestra.run.smithi014.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-06-12T02:38:51.639 INFO:teuthology.orchestra.run.smithi014.stdout:ANSI_COLOR="0;31" 2020-06-12T02:38:51.639 INFO:teuthology.orchestra.run.smithi014.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-06-12T02:38:51.639 INFO:teuthology.orchestra.run.smithi014.stdout:HOME_URL="https://www.centos.org/" 2020-06-12T02:38:51.640 INFO:teuthology.orchestra.run.smithi014.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-06-12T02:38:51.640 INFO:teuthology.orchestra.run.smithi014.stdout: 2020-06-12T02:38:51.650 INFO:teuthology.orchestra.run.smithi014.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-06-12T02:38:51.650 INFO:teuthology.orchestra.run.smithi014.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-06-12T02:38:51.650 INFO:teuthology.orchestra.run.smithi014.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-06-12T02:38:51.651 INFO:teuthology.orchestra.run.smithi014.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-06-12T02:38:51.651 INFO:teuthology.orchestra.run.smithi014.stdout: 2020-06-12T02:38:51.651 INFO:teuthology.lock.ops:Updating smithi014.front.sepia.ceph.com on lock server 2020-06-12T02:38:51.679 INFO:teuthology.orchestra.run.smithi089:> uname -m 2020-06-12T02:38:51.699 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2020-06-12T02:38:51.700 INFO:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2020-06-12T02:38:51.752 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Linux" 2020-06-12T02:38:51.752 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8 (Core)" 2020-06-12T02:38:51.752 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2020-06-12T02:38:51.753 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2020-06-12T02:38:51.753 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8" 2020-06-12T02:38:51.753 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8" 2020-06-12T02:38:51.753 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2020-06-12T02:38:51.754 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2020-06-12T02:38:51.754 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2020-06-12T02:38:51.754 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://www.centos.org/" 2020-06-12T02:38:51.754 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2020-06-12T02:38:51.754 INFO:teuthology.orchestra.run.smithi089.stdout: 2020-06-12T02:38:51.754 INFO:teuthology.orchestra.run.smithi089.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2020-06-12T02:38:51.755 INFO:teuthology.orchestra.run.smithi089.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2020-06-12T02:38:51.755 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2020-06-12T02:38:51.755 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2020-06-12T02:38:51.755 INFO:teuthology.orchestra.run.smithi089.stdout: 2020-06-12T02:38:51.756 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2020-06-12T02:38:51.781 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-06-12T02:38:51.814 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-06-12T02:38:51.825 INFO:teuthology.task.internal:Checking for old test directory... 2020-06-12T02:38:51.826 INFO:teuthology.orchestra.run.smithi014:> test '!' -e /home/ubuntu/cephtest 2020-06-12T02:38:51.838 INFO:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2020-06-12T02:38:51.855 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-06-12T02:38:51.870 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-06-12T02:38:51.871 INFO:teuthology.orchestra.run.smithi014:> test -z $(ls -A /var/lib/ceph) 2020-06-12T02:38:51.891 INFO:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2020-06-12T02:38:51.917 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-06-12T02:38:51.969 INFO:teuthology.run_tasks:Running task kernel... 2020-06-12T02:38:51.990 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'node-exporter.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'prometheus.a': {'kdb': True, 'sha1': 'distro'}, 'node-exporter.b': {'kdb': True, 'sha1': 'distro'}} 2020-06-12T02:38:51.991 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-06-12T02:38:51.992 INFO:teuthology.orchestra.run.smithi014:> uname -r 2020-06-12T02:38:52.008 INFO:teuthology.orchestra.run.smithi014.stdout:4.18.0-147.8.1.el8_1.x86_64 2020-06-12T02:38:52.009 INFO:teuthology.task.kernel:Running kernel on smithi014: 4.18.0-147.8.1.el8_1.x86_64 2020-06-12T02:38:52.009 INFO:teuthology.orchestra.run.smithi014:> sudo yum install -y kernel 2020-06-12T02:38:54.687 INFO:teuthology.orchestra.run.smithi014.stdout:CentOS-8 - AppStream 34 MB/s | 7.0 MB 00:00 2020-06-12T02:38:55.957 INFO:teuthology.orchestra.run.smithi014.stdout:CentOS-8 - Base 23 MB/s | 2.2 MB 00:00 2020-06-12T02:38:56.422 INFO:teuthology.orchestra.run.smithi014.stdout:CentOS-8 - Extras 86 kB/s | 5.5 kB 00:00 2020-06-12T02:38:56.576 INFO:teuthology.orchestra.run.smithi014.stdout:Copr repo for ceph-el8 owned by ktdreyer 483 kB/s | 47 kB 00:00 2020-06-12T02:38:57.100 INFO:teuthology.orchestra.run.smithi014.stdout:Extra Packages for Enterprise Linux 14 MB/s | 6.9 MB 00:00 2020-06-12T02:38:58.605 INFO:teuthology.orchestra.run.smithi014.stdout:lab-extras 3.8 MB/s | 20 kB 00:00 2020-06-12T02:38:59.563 INFO:teuthology.orchestra.run.smithi014.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-06-12T02:38:59.564 INFO:teuthology.orchestra.run.smithi014.stdout:Package kernel-4.18.0-147.8.1.el8_1.x86_64 is already installed. 2020-06-12T02:38:59.715 INFO:teuthology.orchestra.run.smithi014.stdout:Dependencies resolved. 2020-06-12T02:38:59.717 INFO:teuthology.orchestra.run.smithi014.stdout:Nothing to do. 2020-06-12T02:38:59.717 INFO:teuthology.orchestra.run.smithi014.stdout:Complete! 2020-06-12T02:38:59.858 INFO:teuthology.orchestra.run.smithi014:> echo no | sudo yum reinstall kernel || true 2020-06-12T02:39:00.284 INFO:teuthology.orchestra.run.smithi014.stdout:Last metadata expiration check: 0:00:02 ago on Fri 12 Jun 2020 02:38:58 AM UTC. 2020-06-12T02:39:00.645 INFO:teuthology.orchestra.run.smithi014.stdout:Dependencies resolved. 2020-06-12T02:39:00.655 INFO:teuthology.orchestra.run.smithi014.stdout:================================================================================ 2020-06-12T02:39:00.655 INFO:teuthology.orchestra.run.smithi014.stdout: Package Architecture Version Repository Size 2020-06-12T02:39:00.656 INFO:teuthology.orchestra.run.smithi014.stdout:================================================================================ 2020-06-12T02:39:00.656 INFO:teuthology.orchestra.run.smithi014.stdout:Reinstalling: 2020-06-12T02:39:00.656 INFO:teuthology.orchestra.run.smithi014.stdout: kernel x86_64 4.18.0-147.8.1.el8_1 CentOS-Base 1.5 M 2020-06-12T02:39:00.656 INFO:teuthology.orchestra.run.smithi014.stdout: 2020-06-12T02:39:00.656 INFO:teuthology.orchestra.run.smithi014.stdout:Transaction Summary 2020-06-12T02:39:00.657 INFO:teuthology.orchestra.run.smithi014.stdout:================================================================================ 2020-06-12T02:39:00.657 INFO:teuthology.orchestra.run.smithi014.stdout: 2020-06-12T02:39:00.657 INFO:teuthology.orchestra.run.smithi014.stdout:Total download size: 1.5 M 2020-06-12T02:39:00.657 INFO:teuthology.orchestra.run.smithi014.stdout:Installed size: 0 2020-06-12T02:39:00.657 INFO:teuthology.orchestra.run.smithi014.stderr:Operation aborted. 2020-06-12T02:39:00.723 INFO:teuthology.orchestra.run.smithi014.stdout:Is this ok [y/N]: 2020-06-12T02:39:00.723 INFO:teuthology.orchestra.run.smithi014:> sudo yum reinstall -y kernel || true 2020-06-12T02:39:01.141 INFO:teuthology.orchestra.run.smithi014.stdout:Last metadata expiration check: 0:00:03 ago on Fri 12 Jun 2020 02:38:58 AM UTC. 2020-06-12T02:39:01.502 INFO:teuthology.orchestra.run.smithi014.stdout:Dependencies resolved. 2020-06-12T02:39:01.503 INFO:teuthology.orchestra.run.smithi014.stdout:================================================================================ 2020-06-12T02:39:01.504 INFO:teuthology.orchestra.run.smithi014.stdout: Package Architecture Version Repository Size 2020-06-12T02:39:01.504 INFO:teuthology.orchestra.run.smithi014.stdout:================================================================================ 2020-06-12T02:39:01.504 INFO:teuthology.orchestra.run.smithi014.stdout:Reinstalling: 2020-06-12T02:39:01.504 INFO:teuthology.orchestra.run.smithi014.stdout: kernel x86_64 4.18.0-147.8.1.el8_1 CentOS-Base 1.5 M 2020-06-12T02:39:01.505 INFO:teuthology.orchestra.run.smithi014.stdout: 2020-06-12T02:39:01.505 INFO:teuthology.orchestra.run.smithi014.stdout:Transaction Summary 2020-06-12T02:39:01.505 INFO:teuthology.orchestra.run.smithi014.stdout:================================================================================ 2020-06-12T02:39:01.505 INFO:teuthology.orchestra.run.smithi014.stdout: 2020-06-12T02:39:01.505 INFO:teuthology.orchestra.run.smithi014.stdout:Total download size: 1.5 M 2020-06-12T02:39:01.506 INFO:teuthology.orchestra.run.smithi014.stdout:Installed size: 0 2020-06-12T02:39:01.506 INFO:teuthology.orchestra.run.smithi014.stdout:Downloading Packages: 2020-06-12T02:39:01.557 INFO:teuthology.orchestra.run.smithi014.stdout:kernel-4.18.0-147.8.1.el8_1.x86_64.rpm 29 MB/s | 1.5 MB 00:00 2020-06-12T02:39:01.557 INFO:teuthology.orchestra.run.smithi014.stdout:-------------------------------------------------------------------------------- 2020-06-12T02:39:01.558 INFO:teuthology.orchestra.run.smithi014.stdout:Total 29 MB/s | 1.5 MB 00:00 2020-06-12T02:39:01.571 INFO:teuthology.orchestra.run.smithi014.stdout:Running transaction check 2020-06-12T02:39:01.754 INFO:teuthology.orchestra.run.smithi014.stdout:Transaction check succeeded. 2020-06-12T02:39:01.755 INFO:teuthology.orchestra.run.smithi014.stdout:Running transaction test 2020-06-12T02:39:01.812 INFO:teuthology.orchestra.run.smithi014.stdout:Transaction test succeeded. 2020-06-12T02:39:01.813 INFO:teuthology.orchestra.run.smithi014.stdout:Running transaction 2020-06-12T02:39:01.931 INFO:teuthology.orchestra.run.smithi014.stdout: Preparing : 1/1 2020-06-12T02:39:02.074 INFO:teuthology.orchestra.run.smithi014.stdout: Reinstalling : kernel-4.18.0-147.8.1.el8_1.x86_64 1/2 2020-06-12T02:39:02.075 INFO:teuthology.orchestra.run.smithi014.stdout: Cleanup : kernel-4.18.0-147.8.1.el8_1.x86_64 2/2 2020-06-12T02:39:02.298 INFO:teuthology.orchestra.run.smithi014.stdout: Running scriptlet: kernel-4.18.0-147.8.1.el8_1.x86_64 2/2 2020-06-12T02:39:02.298 INFO:teuthology.orchestra.run.smithi014.stdout: Verifying : kernel-4.18.0-147.8.1.el8_1.x86_64 1/2 2020-06-12T02:39:02.589 INFO:teuthology.orchestra.run.smithi014.stdout: Verifying : kernel-4.18.0-147.8.1.el8_1.x86_64 2/2 2020-06-12T02:39:02.589 INFO:teuthology.orchestra.run.smithi014.stdout: 2020-06-12T02:39:02.590 INFO:teuthology.orchestra.run.smithi014.stdout:Reinstalled: 2020-06-12T02:39:02.590 INFO:teuthology.orchestra.run.smithi014.stdout: kernel-4.18.0-147.8.1.el8_1.x86_64 2020-06-12T02:39:02.590 INFO:teuthology.orchestra.run.smithi014.stdout: 2020-06-12T02:39:02.590 INFO:teuthology.orchestra.run.smithi014.stdout:Complete! 2020-06-12T02:39:02.666 INFO:teuthology.orchestra.run.smithi014:> rpm -q kernel --last | head -n 1 2020-06-12T02:39:02.729 INFO:teuthology.orchestra.run.smithi014.stdout:kernel-4.18.0-147.8.1.el8_1.x86_64 Fri 12 Jun 2020 02:39:01 AM UTC 2020-06-12T02:39:02.731 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-147.8.1.el8_1.x86_64 2020-06-12T02:39:02.731 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-06-12T02:39:02.732 INFO:teuthology.orchestra.run.smithi089:> uname -r 2020-06-12T02:39:02.752 INFO:teuthology.orchestra.run.smithi089.stdout:4.18.0-147.8.1.el8_1.x86_64 2020-06-12T02:39:02.753 INFO:teuthology.task.kernel:Running kernel on smithi089: 4.18.0-147.8.1.el8_1.x86_64 2020-06-12T02:39:02.753 INFO:teuthology.orchestra.run.smithi089:> sudo yum install -y kernel 2020-06-12T02:39:05.409 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS-8 - AppStream 39 MB/s | 7.0 MB 00:00 2020-06-12T02:39:06.604 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS-8 - Base 25 MB/s | 2.2 MB 00:00 2020-06-12T02:39:07.032 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS-8 - Extras 86 kB/s | 5.5 kB 00:00 2020-06-12T02:39:07.203 INFO:teuthology.orchestra.run.smithi089.stdout:Copr repo for ceph-el8 owned by ktdreyer 391 kB/s | 47 kB 00:00 2020-06-12T02:39:07.728 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 14 MB/s | 6.9 MB 00:00 2020-06-12T02:39:09.131 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 3.9 MB/s | 20 kB 00:00 2020-06-12T02:39:10.055 INFO:teuthology.orchestra.run.smithi089.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-06-12T02:39:10.055 INFO:teuthology.orchestra.run.smithi089.stdout:Package kernel-4.18.0-147.8.1.el8_1.x86_64 is already installed. 2020-06-12T02:39:10.190 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2020-06-12T02:39:10.191 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2020-06-12T02:39:10.191 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2020-06-12T02:39:10.320 INFO:teuthology.orchestra.run.smithi089:> echo no | sudo yum reinstall kernel || true 2020-06-12T02:39:10.753 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:00:01 ago on Fri 12 Jun 2020 02:39:09 AM UTC. 2020-06-12T02:39:11.094 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2020-06-12T02:39:11.112 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2020-06-12T02:39:11.112 INFO:teuthology.orchestra.run.smithi089.stdout: Package Architecture Version Repository Size 2020-06-12T02:39:11.112 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2020-06-12T02:39:11.112 INFO:teuthology.orchestra.run.smithi089.stdout:Reinstalling: 2020-06-12T02:39:11.113 INFO:teuthology.orchestra.run.smithi089.stdout: kernel x86_64 4.18.0-147.8.1.el8_1 CentOS-Base 1.5 M 2020-06-12T02:39:11.113 INFO:teuthology.orchestra.run.smithi089.stdout: 2020-06-12T02:39:11.113 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2020-06-12T02:39:11.113 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2020-06-12T02:39:11.113 INFO:teuthology.orchestra.run.smithi089.stdout: 2020-06-12T02:39:11.113 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 1.5 M 2020-06-12T02:39:11.114 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 0 2020-06-12T02:39:11.118 INFO:teuthology.orchestra.run.smithi089.stderr:Operation aborted. 2020-06-12T02:39:11.205 INFO:teuthology.orchestra.run.smithi089.stdout:Is this ok [y/N]: 2020-06-12T02:39:11.206 INFO:teuthology.orchestra.run.smithi089:> sudo yum reinstall -y kernel || true 2020-06-12T02:39:11.594 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:00:02 ago on Fri 12 Jun 2020 02:39:09 AM UTC. 2020-06-12T02:39:11.935 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2020-06-12T02:39:11.935 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2020-06-12T02:39:11.936 INFO:teuthology.orchestra.run.smithi089.stdout: Package Architecture Version Repository Size 2020-06-12T02:39:11.936 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2020-06-12T02:39:11.936 INFO:teuthology.orchestra.run.smithi089.stdout:Reinstalling: 2020-06-12T02:39:11.936 INFO:teuthology.orchestra.run.smithi089.stdout: kernel x86_64 4.18.0-147.8.1.el8_1 CentOS-Base 1.5 M 2020-06-12T02:39:11.936 INFO:teuthology.orchestra.run.smithi089.stdout: 2020-06-12T02:39:11.937 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2020-06-12T02:39:11.937 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2020-06-12T02:39:11.937 INFO:teuthology.orchestra.run.smithi089.stdout: 2020-06-12T02:39:11.937 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 1.5 M 2020-06-12T02:39:11.937 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 0 2020-06-12T02:39:11.938 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2020-06-12T02:39:11.986 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-4.18.0-147.8.1.el8_1.x86_64.rpm 31 MB/s | 1.5 MB 00:00 2020-06-12T02:39:11.987 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2020-06-12T02:39:11.987 INFO:teuthology.orchestra.run.smithi089.stdout:Total 30 MB/s | 1.5 MB 00:00 2020-06-12T02:39:12.001 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2020-06-12T02:39:12.202 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2020-06-12T02:39:12.203 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2020-06-12T02:39:12.257 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2020-06-12T02:39:12.257 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2020-06-12T02:39:12.375 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2020-06-12T02:39:12.528 INFO:teuthology.orchestra.run.smithi089.stdout: Reinstalling : kernel-4.18.0-147.8.1.el8_1.x86_64 1/2 2020-06-12T02:39:12.528 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : kernel-4.18.0-147.8.1.el8_1.x86_64 2/2 2020-06-12T02:39:12.732 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-4.18.0-147.8.1.el8_1.x86_64 2/2 2020-06-12T02:39:12.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : kernel-4.18.0-147.8.1.el8_1.x86_64 1/2 2020-06-12T02:39:13.080 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : kernel-4.18.0-147.8.1.el8_1.x86_64 2/2 2020-06-12T02:39:13.081 INFO:teuthology.orchestra.run.smithi089.stdout: 2020-06-12T02:39:13.081 INFO:teuthology.orchestra.run.smithi089.stdout:Reinstalled: 2020-06-12T02:39:13.081 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-4.18.0-147.8.1.el8_1.x86_64 2020-06-12T02:39:13.081 INFO:teuthology.orchestra.run.smithi089.stdout: 2020-06-12T02:39:13.082 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2020-06-12T02:39:13.179 INFO:teuthology.orchestra.run.smithi089:> rpm -q kernel --last | head -n 1 2020-06-12T02:39:13.236 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-4.18.0-147.8.1.el8_1.x86_64 Fri 12 Jun 2020 02:39:12 AM UTC 2020-06-12T02:39:13.238 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-147.8.1.el8_1.x86_64 2020-06-12T02:39:13.238 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-06-12T02:39:13.238 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-06-12T02:39:13.239 INFO:teuthology.orchestra.run.smithi014:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-06-12T02:39:13.281 INFO:teuthology.orchestra.run.smithi014.stdout:ttyS1 2020-06-12T02:39:13.295 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2020-06-12T02:39:13.296 INFO:teuthology.orchestra.run.smithi089:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-06-12T02:39:13.335 INFO:teuthology.orchestra.run.smithi089.stdout:ttyS1 2020-06-12T02:39:13.347 INFO:teuthology.run_tasks:Running task internal.base... 2020-06-12T02:39:13.356 INFO:teuthology.task.internal:Creating test directory... 2020-06-12T02:39:13.357 INFO:teuthology.orchestra.run.smithi014:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-06-12T02:39:13.360 INFO:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-06-12T02:39:13.401 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-06-12T02:39:13.410 INFO:teuthology.run_tasks:Running task internal.archive... 2020-06-12T02:39:13.424 INFO:teuthology.task.internal:Creating archive directory... 2020-06-12T02:39:13.424 INFO:teuthology.orchestra.run.smithi014:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-06-12T02:39:13.427 INFO:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-06-12T02:39:13.474 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-06-12T02:39:13.485 INFO:teuthology.task.internal:Enabling coredump saving... 2020-06-12T02:39:13.485 INFO:teuthology.orchestra.run.smithi014:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-06-12T02:39:13.500 INFO:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-06-12T02:39:13.536 INFO:teuthology.orchestra.run.smithi014.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-06-12T02:39:13.552 INFO:teuthology.orchestra.run.smithi014.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-06-12T02:39:13.553 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-06-12T02:39:13.567 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-06-12T02:39:13.569 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-06-12T02:39:13.581 INFO:teuthology.task.internal:Configuring sudo... 2020-06-12T02:39:13.582 INFO:teuthology.orchestra.run.smithi014:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-06-12T02:39:13.595 INFO:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-06-12T02:39:13.642 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-06-12T02:39:13.654 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-06-12T02:39:13.654 INFO:teuthology.orchestra.run.smithi014:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-06-12T02:39:13.672 INFO:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-06-12T02:39:13.696 INFO:teuthology.orchestra.run.smithi014:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-06-12T02:39:13.761 INFO:teuthology.orchestra.run.smithi014:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-06-12T02:39:13.826 INFO:teuthology.orchestra.run.smithi014:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-06-12T02:39:13.903 INFO:teuthology.orchestra.run.smithi014:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-06-12T02:39:13.959 INFO:teuthology.orchestra.run.smithi014:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-06-12T02:39:14.046 INFO:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-06-12T02:39:14.083 INFO:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-06-12T02:39:14.144 INFO:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-06-12T02:39:14.221 INFO:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-06-12T02:39:14.269 INFO:teuthology.orchestra.run.smithi089:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-06-12T02:39:14.355 INFO:teuthology.orchestra.run.smithi014:> sudo service rsyslog restart 2020-06-12T02:39:14.358 INFO:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2020-06-12T02:39:14.422 INFO:teuthology.orchestra.run.smithi014.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-06-12T02:39:14.458 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-06-12T02:39:14.865 INFO:teuthology.run_tasks:Running task internal.timer... 2020-06-12T02:39:14.880 INFO:teuthology.task.internal:Starting timer... 2020-06-12T02:39:14.882 INFO:teuthology.run_tasks:Running task pcp... 2020-06-12T02:39:14.905 INFO:teuthology.run_tasks:Running task selinux... 2020-06-12T02:39:14.921 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2020-06-12T02:39:14.923 INFO:teuthology.orchestra.run.smithi014:> sudo service auditd rotate 2020-06-12T02:39:15.001 INFO:teuthology.orchestra.run.smithi014.stdout:Rotating logs: [ OK ] 2020-06-12T02:39:15.003 INFO:teuthology.orchestra.run.smithi089:> sudo service auditd rotate 2020-06-12T02:39:15.068 INFO:teuthology.orchestra.run.smithi089.stdout:Rotating logs: [ OK ] 2020-06-12T02:39:15.069 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-06-12T02:39:15.070 INFO:teuthology.orchestra.run.smithi014:> /usr/sbin/getenforce 2020-06-12T02:39:15.093 INFO:teuthology.orchestra.run.smithi014.stdout:Permissive 2020-06-12T02:39:15.093 INFO:teuthology.orchestra.run.smithi089:> /usr/sbin/getenforce 2020-06-12T02:39:15.119 INFO:teuthology.orchestra.run.smithi089.stdout:Permissive 2020-06-12T02:39:15.119 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi014.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi089.front.sepia.ceph.com': 'permissive'} 2020-06-12T02:39:15.120 INFO:teuthology.orchestra.run.smithi014:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-06-12T02:39:15.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-06-12T02:39:15.173 INFO:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-06-12T02:39:15.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-06-12T02:39:15.204 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-06-12T02:39:15.204 INFO:teuthology.orchestra.run.smithi014:> sudo /usr/sbin/setenforce permissive 2020-06-12T02:39:15.265 INFO:teuthology.orchestra.run.smithi089:> sudo /usr/sbin/setenforce permissive 2020-06-12T02:39:15.301 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-06-12T02:39:15.311 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2020-06-12T02:39:15.312 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-06-12T02:39:15.330 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-06-12T02:39:15.332 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi014.front.sepia.ceph.com,smithi089.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-06-12T02:39:15.773 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-06-12T02:39:16.939 INFO:teuthology.task.ansible.out: [WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-06-12T02:39:19.449 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-06-12T02:39:19.466 INFO:teuthology.task.ansible.out:Friday 12 June 2020 02:39:19 +0000 (0:00:00.074) 0:00:00.074 *********** 2020-06-12T02:39:19.490 INFO:teuthology.task.ansible.out:Friday 12 June 2020 02:39:19 +0000 (0:00:00.016) 0:00:00.091 *********** 2020-06-12T02:39:21.054 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi089.front.sepia.ceph.com] 2020-06-12T02:39:21.101 INFO:teuthology.task.ansible.out:ok: [smithi014.front.sepia.ceph.com] 2020-06-12T02:39:21.156 INFO:teuthology.task.ansible.out:Friday 12 June 2020 02:39:21 +0000 (0:00:01.674) 0:00:01.765 *********** 2020-06-12T02:39:21.202 INFO:teuthology.task.ansible.out:Friday 12 June 2020 02:39:21 +0000 (0:00:00.042) 0:00:01.808 *********** 2020-06-12T02:39:21.670 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2020-06-12T02:39:21.672 INFO:teuthology.task.ansible.out:ok: [smithi089.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-06-12T02:39:21.733 INFO:teuthology.task.ansible.out:Friday 12 June 2020 02:39:21 +0000 (0:00:00.534) 0:00:02.343 *********** 2020-06-12T02:39:21.749 INFO:teuthology.task.ansible.out:ok: [smithi014.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-06-12T02:39:21.797 INFO:teuthology.task.ansible.out:Friday 12 June 2020 02:39:21 +0000 (0:00:00.064) 0:00:02.407 *********** 2020-06-12T02:39:22.303 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** ok: [smithi089.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-06-12T02:39:22.358 INFO:teuthology.task.ansible.out:Friday 12 June 2020 02:39:22 +0000 (0:00:00.560) 0:00:02.968 *********** 2020-06-12T02:39:22.368 INFO:teuthology.task.ansible.out:ok: [smithi014.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-06-12T02:39:22.413 INFO:teuthology.task.ansible.out:Friday 12 June 2020 02:39:22 +0000 (0:00:00.055) 0:00:03.023 *********** 2020-06-12T02:39:22.421 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-06-12T02:39:22.425 INFO:teuthology.task.ansible.out:skipping: [smithi089.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-06-12T02:39:22.470 INFO:teuthology.task.ansible.out:Friday 12 June 2020 02:39:22 +0000 (0:00:00.056) 0:00:03.080 *********** 2020-06-12T02:39:22.480 INFO:teuthology.task.ansible.out:skipping: [smithi014.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-06-12T02:39:22.522 INFO:teuthology.task.ansible.out:Friday 12 June 2020 02:39:22 +0000 (0:00:00.051) 0:00:03.132 *********** 2020-06-12T02:39:22.995 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-06-12T02:39:22.997 INFO:teuthology.task.ansible.out:ok: [smithi089.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-06-12T02:39:23.034 INFO:teuthology.task.ansible.out:Friday 12 June 2020 02:39:23 +0000 (0:00:00.512) 0:00:03.644 *********** 2020-06-12T02:39:23.045 INFO:teuthology.task.ansible.out:ok: [smithi014.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-06-12T02:39:23.085 INFO:teuthology.task.ansible.out:Friday 12 June 2020 02:39:23 +0000 (0:00:00.051) 0:00:03.695 *********** 2020-06-12T02:39:23.960 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2020-06-12T02:39:23.962 DEBUG:teuthology.task.console_log:Killing console logger for smithi014 2020-06-12T02:39:23.963 DEBUG:teuthology.task.console_log:Killing console logger for smithi089 2020-06-12T02:39:23.963 DEBUG:teuthology.task.console_log:Killing console logger for smithi014 2020-06-12T02:39:23.963 DEBUG:teuthology.task.console_log:Killing console logger for smithi089 2020-06-12T02:39:23.964 DEBUG:teuthology.exit:Finished running handlers