2021-11-20T15:35:56.218 INFO:root:teuthology version: 1.1.0-6cfddd5e 2021-11-20T15:35:56.220 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2021-11-20_04:38:30-rados-wip-yuriw-master-11.19.21-distro-basic-smithi --owner scheduled_yuriw@teuthology --verbose --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} --archive /home/teuthworker/archive/yuriw-2021-11-20_04:38:30-rados-wip-yuriw-master-11.19.21-distro-basic-smithi/6515653 -- /home/teuthworker/archive/yuriw-2021-11-20_04:38:30-rados-wip-yuriw-master-11.19.21-distro-basic-smithi/6515653/orig.config.yaml 2021-11-20T15:35:56.267 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-11-20T15:35:56.349 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2021-11-20_04:38:30-rados-wip-yuriw-master-11.19.21-distro-basic-smithi/6515653 branch: wip-yuriw-master-11.19.21 description: rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} email: ceph-qa@ceph.io first_in_suite: false job_id: '6515653' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2021-11-20_04:38:30-rados-wip-yuriw-master-11.19.21-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuriw-master-11.19.21 ceph: conf: global: mon election default strategy: 3 osd debug inject dispatch delay duration: 0.1 osd debug inject dispatch delay probability: 0.1 osd_recovery_max_active: 10 osd_recovery_max_single_start: 10 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 9a97ad1bc894bb05b5a58acd3c053b0288a9f4aa ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: sha1: 9a97ad1bc894bb05b5a58acd3c053b0288a9f4aa selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuriw-master-11.19.21 sha1: 5dda7997733e0ccc66d1437ac9b5754be667c2fd owner: scheduled_yuriw@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - mon.a - - osd.4 - osd.5 - osd.6 - osd.7 - mon.b - mgr.x - - osd.8 - osd.9 - osd.10 - osd.11 - mon.c sha1: 9a97ad1bc894bb05b5a58acd3c053b0288a9f4aa sleep_before_teardown: 0 suite: rados suite_branch: wip-yuriw-master-11.19.21 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_5dda7997733e0ccc66d1437ac9b5754be667c2fd/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 5dda7997733e0ccc66d1437ac9b5754be667c2fd targets: smithi018.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDW8Ow9MuPM2Qbah8OB8QGqovrdcxVkHMw1zLs565jcSriR1M0bEPFRQo89Bv1PRGMgUPQqxqpEyV/eg1ujP1ghS1OzucZrKW5Fx202idltWl/YmKlVa0YrbtxBLjkDPNUaWdy+UAIiHqbxhgM2TXLSZxOI0h3FyIpUodJ/jq+UdjLOw55yjWB4n8Pe4N6n/chjzxPllPYaVMbGXzajRQUxEZ+d/D461gMynbSIX0y6GT87aQmiJK8XwdsJGITE8kcAug6k/mwNX6aPbitmlvSAJI4nH54pDlAojnmrFWkx+FxgeheKDNpOqF2zbaBlijt3pcsgb3tZQWfl4A5Cor8nu68j/bGg0gsij8pBbnJL49R6gvxQqSYazct26rFyGw2rP4E8QkO2W7lHQcADhnkpzP2fg0MTNn7hCH04CrHJBq77lixELpsoU7+RC2sS9Ouwr+0siJYeoRxO7JCuB/8yTrYZw1BV6VaTICAlYVBTVPF2P8uOZ51HSVj2X582ieE= smithi122.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3ruUEX/DNHq8vDl1JwX8qzUsHMBXUhF1hTg4tHCh7xFwOPtjZ/HprvPuw/zeyJvpz/fiVAwZ6lqELThlP0zAJjfTAzuvT6d285o2QkmdKE6oCsXE+gnJW6SLLEfvkTCTNMUHx2QaJBRvfNDqDmstb2NX7wu2penHOSdME7Ftd2ews63oobz5JLs4khWFNZ/XdwARTAjbo+qfvVx6xNPWKX295TCvRA3cE2FpEL/477YFlh5vQnivEF0vtyxprcJvxOIo2GlPHro0pmna8khjYzdYpy2DD4sU3awl94RBNKCAQNjNtwf2wP8xEfCSmVP2xbn2oTRXSBXNB4h1eQZpNkoSD/5pdz9EQPNMc7LIYFBPpXX0AnndfNiQM5NPXB1Mi/EeRXUhaWYa/biym82pfx/nPfGgukl1aD0W5F7ByHBuxPUvzKtZokl8dLYkOrO5VP3xPPGbySl7wLenRD+u34Cx/aZ0hH1GRx1g7N5LaODp8Q0r53lGtxU4PEgTtj6M= smithi191.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtJKiN18PwVhkI0rApTOrWTO6r0xJPcE2wcXV/oDEV2CGu3Uzet5QZ6B8vS4VC2PrOy0byafE1695QK/9cPGOpdM8pKbUuR/YQYserIXDrDKj0+16iuAjFNQOowZia+GsozHkClEfwZC1Skx6URxV/wmM2vStUOPrUlZeehoSglbS0tGTa0hg6hHSEHMz/TntsDbQ5cem5VDDkBu4OIr/iaK0R0IkBWOIjMWp2uTZ3DGFtioe9rBfC8U48IFosfL70xRPNnyA2XYlEoxbp7LxRXbqIMcjCxmrwobm3UMBtkmHNkM1JdyZZzWGhy+xlTa2Hwbv0dHZmQH5RCBaGe+BKrpfIphUmtDSJqSGqlT7H5dUFY0pmgG84EP7SZazLOfExJlyljkvYTMPowIGDph0NP127KjrKoU4qXoJ8fBOFYhfPE75/Bhz5cZ1Kf689ap77KlWTjR/bOU6xusUaI4lOx7z9zVNN8BANMBUInUQmoX1n6etBcVqc2Q5vyjkElVs= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: chance_pgnum_grow: 2 chance_pgpnum_fix: 1 min_in: 8 timeout: 1200 - rados: clients: - client.0 ec_pool: true erasure_code_profile: crush-failure-domain: osd k: 4 m: 2 name: jerasure21profile plugin: jerasure technique: reed_sol_van objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 4000 write_append_excl: false teuthology_branch: master teuthology_sha1: 6cfddd5e1c7d5ede1367cf3087c21af44eb88356 timestamp: 2021-11-20_04:38:30 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.20261 2021-11-20T15:35:56.349 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_5dda7997733e0ccc66d1437ac9b5754be667c2fd/qa; will attempt to use it 2021-11-20T15:35:56.350 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_5dda7997733e0ccc66d1437ac9b5754be667c2fd/qa/tasks 2021-11-20T15:35:56.351 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-11-20T15:35:56.353 INFO:teuthology.task.internal:Checking packages... 2021-11-20T15:35:56.377 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9a97ad1bc894bb05b5a58acd3c053b0288a9f4aa' 2021-11-20T15:35:56.378 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-11-20T15:35:56.378 INFO:teuthology.packaging:ref: None 2021-11-20T15:35:56.379 INFO:teuthology.packaging:tag: None 2021-11-20T15:35:56.379 INFO:teuthology.packaging:branch: wip-yuriw-master-11.19.21 2021-11-20T15:35:56.379 INFO:teuthology.packaging:sha1: 9a97ad1bc894bb05b5a58acd3c053b0288a9f4aa 2021-11-20T15:35:56.380 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuriw-master-11.19.21 2021-11-20T15:35:56.600 INFO:teuthology.task.internal:Found packages for ceph version 17.0.0-9147.g5dda7997 2021-11-20T15:35:56.600 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-11-20T15:35:56.612 INFO:teuthology.task.internal:no buildpackages task found 2021-11-20T15:35:56.613 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-11-20T15:35:56.672 INFO:teuthology.task.internal:Saving configuration 2021-11-20T15:35:56.697 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-11-20T15:35:56.734 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-20T15:35:56.757 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi018.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2021-11-20_04:38:30-rados-wip-yuriw-master-11.19.21-distro-basic-smithi/6515653', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2021-11-20 15:30:37.675214', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGL+Wz3/SQf1w5+4ZlS0q7Vne9Vxeei6uDKrdACbrJaF0OqB3OCK3YM9KhaRnMsWBDWdXedK01OZ1YJwIJDDia0='} 2021-11-20T15:35:56.781 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2021-11-20_04:38:30-rados-wip-yuriw-master-11.19.21-distro-basic-smithi/6515653', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2021-11-20 15:30:37.674190', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEvdgS5e+o4XQR5pwbIDZ0gWSYJiBFBw5KlwS2cXDF3zue6wsoLquliNdj0Dk8dy/a1De8/bmrnhblVeRPBAAro='} 2021-11-20T15:35:56.805 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2021-11-20_04:38:30-rados-wip-yuriw-master-11.19.21-distro-basic-smithi/6515653', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2021-11-20 15:30:37.671247', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPuKa2Rg+23TSYcBAu1DiIPq1xcGR1sbHER0NaYorRMUZPffOsHvgrfi/mG2aD4a9/bE9ZoshAV/6DkR+3iao/Q='} 2021-11-20T15:35:56.805 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2021-11-20T15:35:56.815 INFO:teuthology.task.internal:roles: ubuntu@smithi018.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a'] 2021-11-20T15:35:56.816 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x'] 2021-11-20T15:35:56.817 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c'] 2021-11-20T15:35:56.817 INFO:teuthology.run_tasks:Running task console_log... 2021-11-20T15:35:56.989 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1e66714d90>, signals=[15]) 2021-11-20T15:35:56.990 INFO:teuthology.run_tasks:Running task internal.connect... 2021-11-20T15:35:57.000 INFO:teuthology.task.internal:Opening connections... 2021-11-20T15:35:57.000 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi018.front.sepia.ceph.com 2021-11-20T15:35:57.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-20T15:35:57.075 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2021-11-20T15:35:57.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-20T15:35:57.145 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2021-11-20T15:35:57.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-20T15:35:57.216 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2021-11-20T15:35:57.229 DEBUG:teuthology.orchestra.run.smithi018:> uname -m 2021-11-20T15:35:57.250 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64 2021-11-20T15:35:57.251 DEBUG:teuthology.orchestra.run.smithi018:> cat /etc/os-release 2021-11-20T15:35:57.309 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="CentOS Stream" 2021-11-20T15:35:57.309 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="8" 2021-11-20T15:35:57.310 INFO:teuthology.orchestra.run.smithi018.stdout:ID="centos" 2021-11-20T15:35:57.310 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE="rhel fedora" 2021-11-20T15:35:57.310 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="8" 2021-11-20T15:35:57.310 INFO:teuthology.orchestra.run.smithi018.stdout:PLATFORM_ID="platform:el8" 2021-11-20T15:35:57.311 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="CentOS Stream 8" 2021-11-20T15:35:57.311 INFO:teuthology.orchestra.run.smithi018.stdout:ANSI_COLOR="0;31" 2021-11-20T15:35:57.311 INFO:teuthology.orchestra.run.smithi018.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-11-20T15:35:57.311 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://centos.org/" 2021-11-20T15:35:57.312 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-20T15:35:57.312 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-20T15:35:57.312 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-11-20T15:35:57.313 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2021-11-20T15:35:57.333 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2021-11-20T15:35:57.351 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2021-11-20T15:35:57.352 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2021-11-20T15:35:57.410 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2021-11-20T15:35:57.411 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="8" 2021-11-20T15:35:57.411 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2021-11-20T15:35:57.411 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2021-11-20T15:35:57.411 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="8" 2021-11-20T15:35:57.412 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el8" 2021-11-20T15:35:57.412 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 8" 2021-11-20T15:35:57.412 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2021-11-20T15:35:57.412 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-11-20T15:35:57.412 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2021-11-20T15:35:57.413 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-20T15:35:57.413 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-20T15:35:57.413 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-11-20T15:35:57.413 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2021-11-20T15:35:57.438 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2021-11-20T15:35:57.458 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2021-11-20T15:35:57.458 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2021-11-20T15:35:57.516 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2021-11-20T15:35:57.517 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8" 2021-11-20T15:35:57.517 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2021-11-20T15:35:57.517 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2021-11-20T15:35:57.517 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8" 2021-11-20T15:35:57.518 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8" 2021-11-20T15:35:57.519 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 8" 2021-11-20T15:35:57.519 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2021-11-20T15:35:57.519 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-11-20T15:35:57.519 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2021-11-20T15:35:57.519 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-20T15:35:57.520 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-20T15:35:57.520 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-11-20T15:35:57.521 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2021-11-20T15:35:57.539 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2021-11-20T15:35:57.551 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2021-11-20T15:35:57.598 INFO:teuthology.task.internal:Checking for old test directory... 2021-11-20T15:35:57.599 DEBUG:teuthology.orchestra.run.smithi018:> test '!' -e /home/ubuntu/cephtest 2021-11-20T15:35:57.603 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2021-11-20T15:35:57.606 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2021-11-20T15:35:57.624 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2021-11-20T15:35:57.653 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2021-11-20T15:35:57.654 DEBUG:teuthology.orchestra.run.smithi018:> test -z $(ls -A /var/lib/ceph) 2021-11-20T15:35:57.662 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2021-11-20T15:35:57.664 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2021-11-20T15:35:57.698 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2021-11-20T15:35:57.795 INFO:teuthology.run_tasks:Running task kernel... 2021-11-20T15:35:57.830 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2021-11-20T15:35:57.831 INFO:teuthology.task.kernel:config {'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.8': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2021-11-20T15:35:57.831 DEBUG:teuthology.orchestra.run.smithi018:> uname -r 2021-11-20T15:35:57.833 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2021-11-20T15:35:57.834 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2021-11-20T15:35:57.852 INFO:teuthology.orchestra.run.smithi122.stdout:4.18.0-301.1.el8.x86_64 2021-11-20T15:35:57.852 INFO:teuthology.task.kernel:Running kernel on smithi122: 4.18.0-301.1.el8.x86_64 2021-11-20T15:35:57.853 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum install -y kernel 2021-11-20T15:35:57.855 INFO:teuthology.orchestra.run.smithi018.stdout:4.18.0-301.1.el8.x86_64 2021-11-20T15:35:57.855 INFO:teuthology.orchestra.run.smithi191.stdout:4.18.0-301.1.el8.x86_64 2021-11-20T15:35:57.855 INFO:teuthology.task.kernel:Running kernel on smithi018: 4.18.0-301.1.el8.x86_64 2021-11-20T15:35:57.856 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum install -y kernel 2021-11-20T15:35:57.856 INFO:teuthology.task.kernel:Running kernel on smithi191: 4.18.0-301.1.el8.x86_64 2021-11-20T15:35:57.857 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum install -y kernel 2021-11-20T15:36:00.611 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-8 - AppStream 49 MB/s | 17 MB 00:00 2021-11-20T15:36:00.736 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - AppStream 41 MB/s | 17 MB 00:00 2021-11-20T15:36:00.890 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS-8 - AppStream 37 MB/s | 17 MB 00:00 2021-11-20T15:36:04.088 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-8 - Base 25 MB/s | 15 MB 00:00 2021-11-20T15:36:05.147 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS-8 - Base 11 MB/s | 15 MB 00:01 2021-11-20T15:36:05.286 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - Base 10 MB/s | 15 MB 00:01 2021-11-20T15:36:06.298 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-8 - Extras 223 kB/s | 16 kB 00:00 2021-11-20T15:36:07.156 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 14 MB/s | 11 MB 00:00 2021-11-20T15:36:07.508 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS-8 - Extras 197 kB/s | 16 kB 00:00 2021-11-20T15:36:07.627 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS-8 - Extras 218 kB/s | 16 kB 00:00 2021-11-20T15:36:08.684 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 10 MB/s | 11 MB 00:01 2021-11-20T15:36:09.379 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 6.6 MB/s | 11 MB 00:01 2021-11-20T15:36:09.600 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2021-11-20T15:36:11.175 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 4.0 MB/s | 24 kB 00:00 2021-11-20T15:36:11.591 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-4.18.0-240.22.1.el8_3.x86_64 is already installed. 2021-11-20T15:36:11.591 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-4.18.0-301.1.el8.x86_64 is already installed. 2021-11-20T15:36:11.592 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-4.18.0-240.el8.x86_64 is already installed. 2021-11-20T15:36:11.919 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2021-11-20T15:36:11.920 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2021-11-20T15:36:11.920 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2021-11-20T15:36:11.921 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2021-11-20T15:36:11.921 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2021-11-20T15:36:11.921 INFO:teuthology.orchestra.run.smithi122.stdout: kernel x86_64 4.18.0-348.2.1.el8_5 CentOS-Base 7.0 M 2021-11-20T15:36:11.921 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-core x86_64 4.18.0-348.2.1.el8_5 CentOS-Base 38 M 2021-11-20T15:36:11.922 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules x86_64 4.18.0-348.2.1.el8_5 CentOS-Base 30 M 2021-11-20T15:36:11.922 INFO:teuthology.orchestra.run.smithi122.stdout:Removing: 2021-11-20T15:36:11.922 INFO:teuthology.orchestra.run.smithi122.stdout: kernel x86_64 4.18.0-240.el8 @anaconda 0 2021-11-20T15:36:11.922 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-core x86_64 4.18.0-240.el8 @anaconda 62 M 2021-11-20T15:36:11.923 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules x86_64 4.18.0-240.el8 @anaconda 21 M 2021-11-20T15:36:11.923 INFO:teuthology.orchestra.run.smithi122.stdout: 2021-11-20T15:36:11.923 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2021-11-20T15:36:11.923 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2021-11-20T15:36:11.923 INFO:teuthology.orchestra.run.smithi122.stdout:Install 3 Packages 2021-11-20T15:36:11.924 INFO:teuthology.orchestra.run.smithi122.stdout:Remove 3 Packages 2021-11-20T15:36:11.924 INFO:teuthology.orchestra.run.smithi122.stdout: 2021-11-20T15:36:11.924 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 74 M 2021-11-20T15:36:11.925 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2021-11-20T15:36:11.941 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2021-11-20T15:36:12.152 INFO:teuthology.orchestra.run.smithi122.stdout:(1/3): kernel-4.18.0-348.2.1.el8_5.x86_64.rpm 30 MB/s | 7.0 MB 00:00 2021-11-20T15:36:12.864 INFO:teuthology.orchestra.run.smithi122.stdout:(2/3): kernel-core-4.18.0-348.2.1.el8_5.x86_64. 40 MB/s | 38 MB 00:00 2021-11-20T15:36:13.164 INFO:teuthology.orchestra.run.smithi122.stdout:(3/3): kernel-modules-4.18.0-348.2.1.el8_5.x86_ 24 MB/s | 30 MB 00:01 2021-11-20T15:36:13.168 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2021-11-20T15:36:13.168 INFO:teuthology.orchestra.run.smithi122.stdout:Total 60 MB/s | 74 MB 00:01 2021-11-20T15:36:13.201 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-240.22.1.el8_3.x86_64 is already installed. 2021-11-20T15:36:13.201 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-301.1.el8.x86_64 is already installed. 2021-11-20T15:36:13.201 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-240.el8.x86_64 is already installed. 2021-11-20T15:36:13.298 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2021-11-20T15:36:13.527 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2021-11-20T15:36:13.528 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2021-11-20T15:36:13.529 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2021-11-20T15:36:13.529 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2021-11-20T15:36:13.529 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2021-11-20T15:36:13.529 INFO:teuthology.orchestra.run.smithi191.stdout: kernel x86_64 4.18.0-348.2.1.el8_5 CentOS-Base 7.0 M 2021-11-20T15:36:13.530 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-core x86_64 4.18.0-348.2.1.el8_5 CentOS-Base 38 M 2021-11-20T15:36:13.530 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules x86_64 4.18.0-348.2.1.el8_5 CentOS-Base 30 M 2021-11-20T15:36:13.531 INFO:teuthology.orchestra.run.smithi191.stdout:Removing: 2021-11-20T15:36:13.531 INFO:teuthology.orchestra.run.smithi191.stdout: kernel x86_64 4.18.0-240.el8 @anaconda 0 2021-11-20T15:36:13.532 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-core x86_64 4.18.0-240.el8 @anaconda 62 M 2021-11-20T15:36:13.532 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules x86_64 4.18.0-240.el8 @anaconda 21 M 2021-11-20T15:36:13.532 INFO:teuthology.orchestra.run.smithi191.stdout: 2021-11-20T15:36:13.533 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2021-11-20T15:36:13.533 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2021-11-20T15:36:13.533 INFO:teuthology.orchestra.run.smithi191.stdout:Install 3 Packages 2021-11-20T15:36:13.533 INFO:teuthology.orchestra.run.smithi191.stdout:Remove 3 Packages 2021-11-20T15:36:13.533 INFO:teuthology.orchestra.run.smithi191.stdout: 2021-11-20T15:36:13.536 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 74 M 2021-11-20T15:36:13.536 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2021-11-20T15:36:13.687 INFO:teuthology.orchestra.run.smithi191.stdout:(1/3): kernel-4.18.0-348.2.1.el8_5.x86_64.rpm 45 MB/s | 7.0 MB 00:00 2021-11-20T15:36:13.941 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2021-11-20T15:36:13.941 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2021-11-20T15:36:14.126 INFO:teuthology.orchestra.run.smithi018.stdout:Package kernel-4.18.0-240.22.1.el8_3.x86_64 is already installed. 2021-11-20T15:36:14.126 INFO:teuthology.orchestra.run.smithi018.stdout:Package kernel-4.18.0-301.1.el8.x86_64 is already installed. 2021-11-20T15:36:14.126 INFO:teuthology.orchestra.run.smithi018.stdout:Package kernel-4.18.0-240.el8.x86_64 is already installed. 2021-11-20T15:36:14.492 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2021-11-20T15:36:14.494 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2021-11-20T15:36:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2021-11-20T15:36:14.495 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2021-11-20T15:36:14.495 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2021-11-20T15:36:14.495 INFO:teuthology.orchestra.run.smithi018.stdout: kernel x86_64 4.18.0-348.2.1.el8_5 CentOS-Base 7.0 M 2021-11-20T15:36:14.496 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-core x86_64 4.18.0-348.2.1.el8_5 CentOS-Base 38 M 2021-11-20T15:36:14.496 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules x86_64 4.18.0-348.2.1.el8_5 CentOS-Base 30 M 2021-11-20T15:36:14.496 INFO:teuthology.orchestra.run.smithi018.stdout:Removing: 2021-11-20T15:36:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: kernel x86_64 4.18.0-240.el8 @anaconda 0 2021-11-20T15:36:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-core x86_64 4.18.0-240.el8 @anaconda 62 M 2021-11-20T15:36:14.497 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules x86_64 4.18.0-240.el8 @anaconda 21 M 2021-11-20T15:36:14.498 INFO:teuthology.orchestra.run.smithi018.stdout: 2021-11-20T15:36:14.498 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2021-11-20T15:36:14.498 INFO:teuthology.orchestra.run.smithi018.stdout:================================================================================ 2021-11-20T15:36:14.498 INFO:teuthology.orchestra.run.smithi018.stdout:Install 3 Packages 2021-11-20T15:36:14.499 INFO:teuthology.orchestra.run.smithi018.stdout:Remove 3 Packages 2021-11-20T15:36:14.499 INFO:teuthology.orchestra.run.smithi018.stdout: 2021-11-20T15:36:14.499 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 74 M 2021-11-20T15:36:14.499 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2021-11-20T15:36:14.566 INFO:teuthology.orchestra.run.smithi191.stdout:(2/3): kernel-core-4.18.0-348.2.1.el8_5.x86_64. 36 MB/s | 38 MB 00:01 2021-11-20T15:36:14.772 INFO:teuthology.orchestra.run.smithi018.stdout:(1/3): kernel-4.18.0-348.2.1.el8_5.x86_64.rpm 25 MB/s | 7.0 MB 00:00 2021-11-20T15:36:14.838 INFO:teuthology.orchestra.run.smithi191.stdout:(3/3): kernel-modules-4.18.0-348.2.1.el8_5.x86_ 23 MB/s | 30 MB 00:01 2021-11-20T15:36:14.843 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2021-11-20T15:36:14.843 INFO:teuthology.orchestra.run.smithi191.stdout:Total 57 MB/s | 74 MB 00:01 2021-11-20T15:36:14.975 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2021-11-20T15:36:15.036 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2021-11-20T15:36:15.059 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2021-11-20T15:36:15.479 INFO:teuthology.orchestra.run.smithi018.stdout:(2/3): kernel-core-4.18.0-348.2.1.el8_5.x86_64. 38 MB/s | 38 MB 00:00 2021-11-20T15:36:15.648 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2021-11-20T15:36:15.648 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2021-11-20T15:36:15.837 INFO:teuthology.orchestra.run.smithi018.stdout:(3/3): kernel-modules-4.18.0-348.2.1.el8_5.x86_ 22 MB/s | 30 MB 00:01 2021-11-20T15:36:15.843 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2021-11-20T15:36:15.843 INFO:teuthology.orchestra.run.smithi018.stdout:Total 55 MB/s | 74 MB 00:01 2021-11-20T15:36:15.978 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2021-11-20T15:36:16.690 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2021-11-20T15:36:16.690 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2021-11-20T15:36:16.802 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2021-11-20T15:36:16.823 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2021-11-20T15:36:17.707 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2021-11-20T15:36:17.812 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2021-11-20T15:36:17.834 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2021-11-20T15:36:18.243 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-core-4.18.0-348.2.1.el8_5.x86_64 1/6 2021-11-20T15:36:19.494 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2021-11-20T15:36:19.983 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-core-4.18.0-348.2.1.el8_5.x86_64 1/6 2021-11-20T15:36:20.666 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2021-11-20T15:36:21.176 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-core-4.18.0-348.2.1.el8_5.x86_64 1/6 2021-11-20T15:36:22.493 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-4.18.0-348.2.1.el8_5.x86_64 1/6 2021-11-20T15:36:22.909 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-modules-4.18.0-348.2.1.el8_5.x86_64 2/6 2021-11-20T15:36:24.290 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-core-4.18.0-348.2.1.el8_5.x86_64 1/6 2021-11-20T15:36:24.691 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-modules-4.18.0-348.2.1.el8_5.x86_64 2/6 2021-11-20T15:36:25.611 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-core-4.18.0-348.2.1.el8_5.x86_64 1/6 2021-11-20T15:36:25.971 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-4.18.0-348.2.1.el8_5.x86_64 2/6 2021-11-20T15:36:25.974 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-modules-4.18.0-348.2.1.el8_5.x86_64 2/6 2021-11-20T15:36:26.171 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-4.18.0-348.2.1.el8_5.x86_64 3/6 2021-11-20T15:36:26.172 INFO:teuthology.orchestra.run.smithi122.stdout: Erasing : kernel-4.18.0-240.el8.x86_64 4/6 2021-11-20T15:36:26.412 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-4.18.0-240.el8.x86_64 4/6 2021-11-20T15:36:26.412 INFO:teuthology.orchestra.run.smithi122.stdout: Erasing : kernel-modules-4.18.0-240.el8.x86_64 5/6 2021-11-20T15:36:27.753 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-modules-4.18.0-348.2.1.el8_5.x86_64 2/6 2021-11-20T15:36:27.928 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-4.18.0-348.2.1.el8_5.x86_64 3/6 2021-11-20T15:36:27.929 INFO:teuthology.orchestra.run.smithi191.stdout: Erasing : kernel-4.18.0-240.el8.x86_64 4/6 2021-11-20T15:36:28.166 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-4.18.0-240.el8.x86_64 4/6 2021-11-20T15:36:28.166 INFO:teuthology.orchestra.run.smithi191.stdout: Erasing : kernel-modules-4.18.0-240.el8.x86_64 5/6 2021-11-20T15:36:28.632 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-4.18.0-240.el8.x86_64 5/6 2021-11-20T15:36:28.822 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-4.18.0-240.el8.x86_64 6/6 2021-11-20T15:36:28.832 INFO:teuthology.orchestra.run.smithi122.stdout: Erasing : kernel-core-4.18.0-240.el8.x86_64 6/6 2021-11-20T15:36:29.266 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-modules-4.18.0-348.2.1.el8_5.x86_64 2/6 2021-11-20T15:36:29.471 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : kernel-4.18.0-348.2.1.el8_5.x86_64 3/6 2021-11-20T15:36:29.472 INFO:teuthology.orchestra.run.smithi018.stdout: Erasing : kernel-4.18.0-240.el8.x86_64 4/6 2021-11-20T15:36:29.718 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-4.18.0-240.el8.x86_64 4/6 2021-11-20T15:36:29.718 INFO:teuthology.orchestra.run.smithi018.stdout: Erasing : kernel-modules-4.18.0-240.el8.x86_64 5/6 2021-11-20T15:36:30.323 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-modules-4.18.0-240.el8.x86_64 5/6 2021-11-20T15:36:30.544 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-core-4.18.0-240.el8.x86_64 6/6 2021-11-20T15:36:30.553 INFO:teuthology.orchestra.run.smithi191.stdout: Erasing : kernel-core-4.18.0-240.el8.x86_64 6/6 2021-11-20T15:36:31.874 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-modules-4.18.0-240.el8.x86_64 5/6 2021-11-20T15:36:31.936 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-4.18.0-240.el8.x86_64 6/6 2021-11-20T15:36:32.103 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-core-4.18.0-240.el8.x86_64 6/6 2021-11-20T15:36:32.113 INFO:teuthology.orchestra.run.smithi018.stdout: Erasing : kernel-core-4.18.0-240.el8.x86_64 6/6 2021-11-20T15:36:33.517 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-core-4.18.0-240.el8.x86_64 6/6 2021-11-20T15:36:35.120 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-core-4.18.0-240.el8.x86_64 6/6 2021-11-20T15:36:52.626 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-4.18.0-348.2.1.el8_5.x86_64 6/6 2021-11-20T15:36:53.014 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-4.18.0-240.el8.x86_64 6/6 2021-11-20T15:36:53.014 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-4.18.0-348.2.1.el8_5.x86_64 1/6 2021-11-20T15:36:53.015 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-core-4.18.0-348.2.1.el8_5.x86_64 2/6 2021-11-20T15:36:53.015 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-modules-4.18.0-348.2.1.el8_5.x86_64 3/6 2021-11-20T15:36:53.015 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-4.18.0-240.el8.x86_64 4/6 2021-11-20T15:36:53.015 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-core-4.18.0-240.el8.x86_64 5/6 2021-11-20T15:36:53.547 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-modules-4.18.0-240.el8.x86_64 6/6 2021-11-20T15:36:53.547 INFO:teuthology.orchestra.run.smithi122.stdout: 2021-11-20T15:36:53.547 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2021-11-20T15:36:53.548 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:53.548 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-core-4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:53.548 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:53.548 INFO:teuthology.orchestra.run.smithi122.stdout:Removed: 2021-11-20T15:36:53.549 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-4.18.0-240.el8.x86_64 kernel-core-4.18.0-240.el8.x86_64 2021-11-20T15:36:53.549 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-4.18.0-240.el8.x86_64 2021-11-20T15:36:53.549 INFO:teuthology.orchestra.run.smithi122.stdout: 2021-11-20T15:36:53.549 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2021-11-20T15:36:54.159 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-core-4.18.0-348.2.1.el8_5.x86_64 6/6 2021-11-20T15:36:54.345 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel --last | head -n 1 2021-11-20T15:36:54.542 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-core-4.18.0-240.el8.x86_64 6/6 2021-11-20T15:36:54.543 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-4.18.0-348.2.1.el8_5.x86_64 1/6 2021-11-20T15:36:54.543 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-core-4.18.0-348.2.1.el8_5.x86_64 2/6 2021-11-20T15:36:54.543 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-modules-4.18.0-348.2.1.el8_5.x86_64 3/6 2021-11-20T15:36:54.543 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-4.18.0-240.el8.x86_64 4/6 2021-11-20T15:36:54.544 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-core-4.18.0-240.el8.x86_64 5/6 2021-11-20T15:36:54.546 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-4.18.0-348.2.1.el8_5.x86_64 Sat 20 Nov 2021 03:36:25 PM UTC 2021-11-20T15:36:54.546 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:54.546 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-301.1.el8.x86_64 Expected: 4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:54.547 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2021-11-20T15:36:54.547 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-11-20T15:36:54.547 INFO:teuthology.task.kernel:Installing distro kernel on osd.4... 2021-11-20T15:36:54.548 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi122.front.sepia.ceph.com, path=None, version=distro) 2021-11-20T15:36:54.548 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel --last | head -n 1 2021-11-20T15:36:54.690 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-4.18.0-348.2.1.el8_5.x86_64 Sat 20 Nov 2021 03:36:25 PM UTC 2021-11-20T15:36:54.691 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:54.692 DEBUG:teuthology.orchestra.run.smithi122:> sudo rpm -qi grub2-tools 2021-11-20T15:36:54.721 INFO:teuthology.orchestra.run.smithi122.stdout:Name : grub2-tools 2021-11-20T15:36:54.722 INFO:teuthology.orchestra.run.smithi122.stdout:Epoch : 1 2021-11-20T15:36:54.722 INFO:teuthology.orchestra.run.smithi122.stdout:Version : 2.02 2021-11-20T15:36:54.722 INFO:teuthology.orchestra.run.smithi122.stdout:Release : 99.el8 2021-11-20T15:36:54.722 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: x86_64 2021-11-20T15:36:54.723 INFO:teuthology.orchestra.run.smithi122.stdout:Install Date: Thu 13 May 2021 06:08:01 PM UTC 2021-11-20T15:36:54.723 INFO:teuthology.orchestra.run.smithi122.stdout:Group : System Environment/Base 2021-11-20T15:36:54.723 INFO:teuthology.orchestra.run.smithi122.stdout:Size : 9264704 2021-11-20T15:36:54.723 INFO:teuthology.orchestra.run.smithi122.stdout:License : GPLv3+ 2021-11-20T15:36:54.724 INFO:teuthology.orchestra.run.smithi122.stdout:Signature : RSA/SHA256, Tue 02 Mar 2021 09:11:55 PM UTC, Key ID 05b555b38483c65d 2021-11-20T15:36:54.724 INFO:teuthology.orchestra.run.smithi122.stdout:Source RPM : grub2-2.02-99.el8.src.rpm 2021-11-20T15:36:54.724 INFO:teuthology.orchestra.run.smithi122.stdout:Build Date : Tue 02 Mar 2021 09:03:27 PM UTC 2021-11-20T15:36:54.724 INFO:teuthology.orchestra.run.smithi122.stdout:Build Host : kbuilder.bsys.centos.org 2021-11-20T15:36:54.724 INFO:teuthology.orchestra.run.smithi122.stdout:Relocations : (not relocatable) 2021-11-20T15:36:54.725 INFO:teuthology.orchestra.run.smithi122.stdout:Packager : CentOS BuildSystem 2021-11-20T15:36:54.725 INFO:teuthology.orchestra.run.smithi122.stdout:Vendor : CentOS 2021-11-20T15:36:54.725 INFO:teuthology.orchestra.run.smithi122.stdout:URL : http://www.gnu.org/software/grub/ 2021-11-20T15:36:54.725 INFO:teuthology.orchestra.run.smithi122.stdout:Summary : Support tools for GRUB. 2021-11-20T15:36:54.726 INFO:teuthology.orchestra.run.smithi122.stdout:Description : 2021-11-20T15:36:54.726 INFO:teuthology.orchestra.run.smithi122.stdout: 2021-11-20T15:36:54.726 INFO:teuthology.orchestra.run.smithi122.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2021-11-20T15:36:54.726 INFO:teuthology.orchestra.run.smithi122.stdout:customizable bootloader with modular architecture. It supports a rich 2021-11-20T15:36:54.726 INFO:teuthology.orchestra.run.smithi122.stdout:variety of kernel formats, file systems, computer architectures and 2021-11-20T15:36:54.727 INFO:teuthology.orchestra.run.smithi122.stdout:hardware devices. 2021-11-20T15:36:54.727 INFO:teuthology.orchestra.run.smithi122.stdout: 2021-11-20T15:36:54.727 INFO:teuthology.orchestra.run.smithi122.stdout:This subpackage provides tools for support of all platforms. 2021-11-20T15:36:54.728 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2021-11-20T15:36:54.728 INFO:teuthology.task.kernel:Updating grub on smithi122 to boot 4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:54.729 DEBUG:teuthology.orchestra.run.smithi122:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2021-11-20T15:36:54.999 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-modules-4.18.0-240.el8.x86_64 6/6 2021-11-20T15:36:54.999 INFO:teuthology.orchestra.run.smithi191.stdout: 2021-11-20T15:36:55.000 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2021-11-20T15:36:55.000 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:55.000 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-core-4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:55.000 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules-4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:55.001 INFO:teuthology.orchestra.run.smithi191.stdout:Removed: 2021-11-20T15:36:55.001 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-4.18.0-240.el8.x86_64 kernel-core-4.18.0-240.el8.x86_64 2021-11-20T15:36:55.001 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules-4.18.0-240.el8.x86_64 2021-11-20T15:36:55.001 INFO:teuthology.orchestra.run.smithi191.stdout: 2021-11-20T15:36:55.002 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2021-11-20T15:36:55.111 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel --last | head -n 1 2021-11-20T15:36:55.276 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-4.18.0-348.2.1.el8_5.x86_64 Sat 20 Nov 2021 03:36:27 PM UTC 2021-11-20T15:36:55.278 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:55.278 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-301.1.el8.x86_64 Expected: 4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:55.279 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2021-11-20T15:36:55.279 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-11-20T15:36:55.279 INFO:teuthology.task.kernel:Installing distro kernel on osd.8... 2021-11-20T15:36:55.279 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi191.front.sepia.ceph.com, path=None, version=distro) 2021-11-20T15:36:55.280 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel --last | head -n 1 2021-11-20T15:36:55.422 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-4.18.0-348.2.1.el8_5.x86_64 Sat 20 Nov 2021 03:36:27 PM UTC 2021-11-20T15:36:55.424 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:55.424 DEBUG:teuthology.orchestra.run.smithi191:> sudo rpm -qi grub2-tools 2021-11-20T15:36:55.455 INFO:teuthology.orchestra.run.smithi191.stdout:Name : grub2-tools 2021-11-20T15:36:55.455 INFO:teuthology.orchestra.run.smithi191.stdout:Epoch : 1 2021-11-20T15:36:55.455 INFO:teuthology.orchestra.run.smithi191.stdout:Version : 2.02 2021-11-20T15:36:55.456 INFO:teuthology.orchestra.run.smithi191.stdout:Release : 99.el8 2021-11-20T15:36:55.456 INFO:teuthology.orchestra.run.smithi191.stdout:Architecture: x86_64 2021-11-20T15:36:55.456 INFO:teuthology.orchestra.run.smithi191.stdout:Install Date: Thu 13 May 2021 06:08:01 PM UTC 2021-11-20T15:36:55.456 INFO:teuthology.orchestra.run.smithi191.stdout:Group : System Environment/Base 2021-11-20T15:36:55.457 INFO:teuthology.orchestra.run.smithi191.stdout:Size : 9264704 2021-11-20T15:36:55.457 INFO:teuthology.orchestra.run.smithi191.stdout:License : GPLv3+ 2021-11-20T15:36:55.457 INFO:teuthology.orchestra.run.smithi191.stdout:Signature : RSA/SHA256, Tue 02 Mar 2021 09:11:55 PM UTC, Key ID 05b555b38483c65d 2021-11-20T15:36:55.457 INFO:teuthology.orchestra.run.smithi191.stdout:Source RPM : grub2-2.02-99.el8.src.rpm 2021-11-20T15:36:55.457 INFO:teuthology.orchestra.run.smithi191.stdout:Build Date : Tue 02 Mar 2021 09:03:27 PM UTC 2021-11-20T15:36:55.458 INFO:teuthology.orchestra.run.smithi191.stdout:Build Host : kbuilder.bsys.centos.org 2021-11-20T15:36:55.458 INFO:teuthology.orchestra.run.smithi191.stdout:Relocations : (not relocatable) 2021-11-20T15:36:55.458 INFO:teuthology.orchestra.run.smithi191.stdout:Packager : CentOS BuildSystem 2021-11-20T15:36:55.458 INFO:teuthology.orchestra.run.smithi191.stdout:Vendor : CentOS 2021-11-20T15:36:55.459 INFO:teuthology.orchestra.run.smithi191.stdout:URL : http://www.gnu.org/software/grub/ 2021-11-20T15:36:55.459 INFO:teuthology.orchestra.run.smithi191.stdout:Summary : Support tools for GRUB. 2021-11-20T15:36:55.459 INFO:teuthology.orchestra.run.smithi191.stdout:Description : 2021-11-20T15:36:55.459 INFO:teuthology.orchestra.run.smithi191.stdout: 2021-11-20T15:36:55.460 INFO:teuthology.orchestra.run.smithi191.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2021-11-20T15:36:55.460 INFO:teuthology.orchestra.run.smithi191.stdout:customizable bootloader with modular architecture. It supports a rich 2021-11-20T15:36:55.460 INFO:teuthology.orchestra.run.smithi191.stdout:variety of kernel formats, file systems, computer architectures and 2021-11-20T15:36:55.461 INFO:teuthology.orchestra.run.smithi191.stdout:hardware devices. 2021-11-20T15:36:55.461 INFO:teuthology.orchestra.run.smithi191.stdout: 2021-11-20T15:36:55.461 INFO:teuthology.orchestra.run.smithi191.stdout:This subpackage provides tools for support of all platforms. 2021-11-20T15:36:55.462 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2021-11-20T15:36:55.463 INFO:teuthology.task.kernel:Updating grub on smithi191 to boot 4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:55.463 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2021-11-20T15:36:56.151 INFO:teuthology.orchestra.run.smithi122.stderr:Generating grub configuration file ... 2021-11-20T15:36:57.210 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-core-4.18.0-348.2.1.el8_5.x86_64 6/6 2021-11-20T15:36:57.318 INFO:teuthology.orchestra.run.smithi191.stderr:Generating grub configuration file ... 2021-11-20T15:36:57.604 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: kernel-core-4.18.0-240.el8.x86_64 6/6 2021-11-20T15:36:57.605 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-4.18.0-348.2.1.el8_5.x86_64 1/6 2021-11-20T15:36:57.605 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-core-4.18.0-348.2.1.el8_5.x86_64 2/6 2021-11-20T15:36:57.605 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-modules-4.18.0-348.2.1.el8_5.x86_64 3/6 2021-11-20T15:36:57.605 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-4.18.0-240.el8.x86_64 4/6 2021-11-20T15:36:57.605 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-core-4.18.0-240.el8.x86_64 5/6 2021-11-20T15:36:58.088 INFO:teuthology.orchestra.run.smithi122.stderr:done 2021-11-20T15:36:58.091 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2021-11-20T15:36:58.103 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : kernel-modules-4.18.0-240.el8.x86_64 6/6 2021-11-20T15:36:58.103 INFO:teuthology.orchestra.run.smithi018.stdout: 2021-11-20T15:36:58.103 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2021-11-20T15:36:58.104 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:58.104 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-core-4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:58.104 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules-4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:58.104 INFO:teuthology.orchestra.run.smithi018.stdout:Removed: 2021-11-20T15:36:58.105 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-4.18.0-240.el8.x86_64 kernel-core-4.18.0-240.el8.x86_64 2021-11-20T15:36:58.105 INFO:teuthology.orchestra.run.smithi018.stdout: kernel-modules-4.18.0-240.el8.x86_64 2021-11-20T15:36:58.105 INFO:teuthology.orchestra.run.smithi018.stdout: 2021-11-20T15:36:58.105 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2021-11-20T15:36:58.111 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.VclezP8Y35 2021-11-20T15:36:58.111 DEBUG:teuthology.orchestra.run.smithi122:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VclezP8Y35 2021-11-20T15:36:58.182 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.VclezP8Y35 2021-11-20T15:36:58.236 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q kernel --last | head -n 1 2021-11-20T15:36:58.343 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.VclezP8Y35 is 6KB 2021-11-20T15:36:58.392 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.VclezP8Y35 2021-11-20T15:36:58.407 DEBUG:teuthology.orchestra.run.smithi122:> sudo /bin/ls /boot/loader/entries || true 2021-11-20T15:36:58.442 INFO:teuthology.orchestra.run.smithi018.stdout:kernel-4.18.0-348.2.1.el8_5.x86_64 Sat 20 Nov 2021 03:36:29 PM UTC 2021-11-20T15:36:58.444 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:58.444 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-301.1.el8.x86_64 Expected: 4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:58.444 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2021-11-20T15:36:58.444 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-11-20T15:36:58.445 INFO:teuthology.task.kernel:Installing distro kernel on osd.0... 2021-11-20T15:36:58.445 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi018.front.sepia.ceph.com, path=None, version=distro) 2021-11-20T15:36:58.445 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q kernel --last | head -n 1 2021-11-20T15:36:58.469 INFO:teuthology.orchestra.run.smithi122.stdout:3331cc438d4e47d3b31025a6d3bd39f2-0-rescue.conf 2021-11-20T15:36:58.469 INFO:teuthology.orchestra.run.smithi122.stdout:3331cc438d4e47d3b31025a6d3bd39f2-4.18.0-240.22.1.el8_3.x86_64.conf 2021-11-20T15:36:58.469 INFO:teuthology.orchestra.run.smithi122.stdout:3331cc438d4e47d3b31025a6d3bd39f2-4.18.0-301.1.el8.x86_64.conf 2021-11-20T15:36:58.470 INFO:teuthology.orchestra.run.smithi122.stdout:3331cc438d4e47d3b31025a6d3bd39f2-4.18.0-348.2.1.el8_5.x86_64.conf 2021-11-20T15:36:58.471 DEBUG:teuthology.orchestra.run.smithi122:> sudo grub2-set-default 3331cc438d4e47d3b31025a6d3bd39f2-4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:58.595 INFO:teuthology.orchestra.run.smithi018.stdout:kernel-4.18.0-348.2.1.el8_5.x86_64 Sat 20 Nov 2021 03:36:29 PM UTC 2021-11-20T15:36:58.596 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:58.597 DEBUG:teuthology.orchestra.run.smithi018:> sudo rpm -qi grub2-tools 2021-11-20T15:36:58.628 INFO:teuthology.orchestra.run.smithi018.stdout:Name : grub2-tools 2021-11-20T15:36:58.629 INFO:teuthology.orchestra.run.smithi018.stdout:Epoch : 1 2021-11-20T15:36:58.629 INFO:teuthology.orchestra.run.smithi018.stdout:Version : 2.02 2021-11-20T15:36:58.630 INFO:teuthology.orchestra.run.smithi018.stdout:Release : 99.el8 2021-11-20T15:36:58.630 INFO:teuthology.orchestra.run.smithi018.stdout:Architecture: x86_64 2021-11-20T15:36:58.630 INFO:teuthology.orchestra.run.smithi018.stdout:Install Date: Thu 13 May 2021 06:08:01 PM UTC 2021-11-20T15:36:58.630 INFO:teuthology.orchestra.run.smithi018.stdout:Group : System Environment/Base 2021-11-20T15:36:58.631 INFO:teuthology.orchestra.run.smithi018.stdout:Size : 9264704 2021-11-20T15:36:58.631 INFO:teuthology.orchestra.run.smithi018.stdout:License : GPLv3+ 2021-11-20T15:36:58.631 INFO:teuthology.orchestra.run.smithi018.stdout:Signature : RSA/SHA256, Tue 02 Mar 2021 09:11:55 PM UTC, Key ID 05b555b38483c65d 2021-11-20T15:36:58.631 INFO:teuthology.orchestra.run.smithi018.stdout:Source RPM : grub2-2.02-99.el8.src.rpm 2021-11-20T15:36:58.631 INFO:teuthology.orchestra.run.smithi018.stdout:Build Date : Tue 02 Mar 2021 09:03:27 PM UTC 2021-11-20T15:36:58.632 INFO:teuthology.orchestra.run.smithi018.stdout:Build Host : kbuilder.bsys.centos.org 2021-11-20T15:36:58.632 INFO:teuthology.orchestra.run.smithi018.stdout:Relocations : (not relocatable) 2021-11-20T15:36:58.632 INFO:teuthology.orchestra.run.smithi018.stdout:Packager : CentOS BuildSystem 2021-11-20T15:36:58.632 INFO:teuthology.orchestra.run.smithi018.stdout:Vendor : CentOS 2021-11-20T15:36:58.633 INFO:teuthology.orchestra.run.smithi018.stdout:URL : http://www.gnu.org/software/grub/ 2021-11-20T15:36:58.633 INFO:teuthology.orchestra.run.smithi018.stdout:Summary : Support tools for GRUB. 2021-11-20T15:36:58.633 INFO:teuthology.orchestra.run.smithi018.stdout:Description : 2021-11-20T15:36:58.633 INFO:teuthology.orchestra.run.smithi018.stdout: 2021-11-20T15:36:58.633 INFO:teuthology.orchestra.run.smithi018.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2021-11-20T15:36:58.634 INFO:teuthology.orchestra.run.smithi018.stdout:customizable bootloader with modular architecture. It supports a rich 2021-11-20T15:36:58.634 INFO:teuthology.orchestra.run.smithi018.stdout:variety of kernel formats, file systems, computer architectures and 2021-11-20T15:36:58.634 INFO:teuthology.orchestra.run.smithi018.stdout:hardware devices. 2021-11-20T15:36:58.634 INFO:teuthology.orchestra.run.smithi018.stdout: 2021-11-20T15:36:58.634 INFO:teuthology.orchestra.run.smithi018.stdout:This subpackage provides tools for support of all platforms. 2021-11-20T15:36:58.636 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2021-11-20T15:36:58.636 INFO:teuthology.task.kernel:Updating grub on smithi018 to boot 4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:58.636 DEBUG:teuthology.orchestra.run.smithi018:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2021-11-20T15:36:58.640 DEBUG:teuthology.orchestra.run.smithi122:> sudo shutdown -r now 2021-11-20T15:36:59.143 INFO:teuthology.orchestra.run.smithi191.stderr:done 2021-11-20T15:36:59.143 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2021-11-20T15:36:59.164 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.QmJum4KzLP 2021-11-20T15:36:59.165 DEBUG:teuthology.orchestra.run.smithi191:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QmJum4KzLP 2021-11-20T15:36:59.237 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.QmJum4KzLP 2021-11-20T15:36:59.384 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.QmJum4KzLP is 6KB 2021-11-20T15:36:59.432 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.QmJum4KzLP 2021-11-20T15:36:59.448 DEBUG:teuthology.orchestra.run.smithi191:> sudo /bin/ls /boot/loader/entries || true 2021-11-20T15:36:59.513 INFO:teuthology.orchestra.run.smithi191.stdout:3331cc438d4e47d3b31025a6d3bd39f2-0-rescue.conf 2021-11-20T15:36:59.513 INFO:teuthology.orchestra.run.smithi191.stdout:3331cc438d4e47d3b31025a6d3bd39f2-4.18.0-240.22.1.el8_3.x86_64.conf 2021-11-20T15:36:59.513 INFO:teuthology.orchestra.run.smithi191.stdout:3331cc438d4e47d3b31025a6d3bd39f2-4.18.0-301.1.el8.x86_64.conf 2021-11-20T15:36:59.513 INFO:teuthology.orchestra.run.smithi191.stdout:3331cc438d4e47d3b31025a6d3bd39f2-4.18.0-348.2.1.el8_5.x86_64.conf 2021-11-20T15:36:59.515 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-set-default 3331cc438d4e47d3b31025a6d3bd39f2-4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:36:59.675 DEBUG:teuthology.orchestra.run.smithi191:> sudo shutdown -r now 2021-11-20T15:37:00.053 INFO:teuthology.orchestra.run.smithi018.stderr:Generating grub configuration file ... 2021-11-20T15:37:02.165 INFO:teuthology.orchestra.run.smithi018.stderr:done 2021-11-20T15:37:02.167 DEBUG:teuthology.orchestra.run.smithi018:> mktemp 2021-11-20T15:37:02.217 INFO:teuthology.orchestra.run.smithi018.stdout:/tmp/tmp.j8JyZC3aus 2021-11-20T15:37:02.217 DEBUG:teuthology.orchestra.run.smithi018:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.j8JyZC3aus 2021-11-20T15:37:02.247 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 0666 /tmp/tmp.j8JyZC3aus 2021-11-20T15:37:02.407 DEBUG:teuthology.orchestra.remote:smithi018:/tmp/tmp.j8JyZC3aus is 6KB 2021-11-20T15:37:02.457 DEBUG:teuthology.orchestra.run.smithi018:> rm -fr /tmp/tmp.j8JyZC3aus 2021-11-20T15:37:02.473 DEBUG:teuthology.orchestra.run.smithi018:> sudo /bin/ls /boot/loader/entries || true 2021-11-20T15:37:02.546 INFO:teuthology.orchestra.run.smithi018.stdout:3331cc438d4e47d3b31025a6d3bd39f2-0-rescue.conf 2021-11-20T15:37:02.546 INFO:teuthology.orchestra.run.smithi018.stdout:3331cc438d4e47d3b31025a6d3bd39f2-4.18.0-240.22.1.el8_3.x86_64.conf 2021-11-20T15:37:02.546 INFO:teuthology.orchestra.run.smithi018.stdout:3331cc438d4e47d3b31025a6d3bd39f2-4.18.0-301.1.el8.x86_64.conf 2021-11-20T15:37:02.547 INFO:teuthology.orchestra.run.smithi018.stdout:3331cc438d4e47d3b31025a6d3bd39f2-4.18.0-348.2.1.el8_5.x86_64.conf 2021-11-20T15:37:02.548 DEBUG:teuthology.orchestra.run.smithi018:> sudo grub2-set-default 3331cc438d4e47d3b31025a6d3bd39f2-4.18.0-348.2.1.el8_5.x86_64 2021-11-20T15:37:02.744 DEBUG:teuthology.orchestra.run.smithi018:> sudo shutdown -r now 2021-11-20T15:37:03.324 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2021-11-20T15:37:03.325 DEBUG:teuthology.task.console_log:Killing console logger for smithi018 2021-11-20T15:37:03.326 DEBUG:teuthology.task.console_log:Killing console logger for smithi122 2021-11-20T15:37:03.327 DEBUG:teuthology.task.console_log:Killing console logger for smithi191 2021-11-20T15:37:03.327 DEBUG:teuthology.exit:Finished running handlers