2026-02-12T17:20:19.576 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71 2026-02-12T17:20:19.592 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-12T17:20:19.672 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-12_15:38:14-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/46420 branch: wip-yuri4-testing-2026-02-11-2126 description: rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/{bluestore/{alloc$/{btree} base mem$/{low} onode-segment$/{1M} write$/{random/{compr$/{no$/{no}} random}}}} rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects-localized} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '46420' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-12_15:38:14-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri4-testing-2026-02-11-2126 ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr2: false ms inject socket failures: 5000 ms type: async osd_max_pg_log_entries: 2 osd_min_pg_log_entries: 1 osd_pg_log_trim_min: 0 osd_pool_default_min_size: 2 osd_pool_default_size: 3 osd_recovery_max_active: 10 osd_recovery_max_single_start: 10 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 30 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore allocator: btree bluestore block size: 96636764160 bluestore fsck on mount: true bluestore onode segment size: 1024K bluestore write v2 random: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd heartbeat use min delay socket: true osd max backfills: 3 osd max markdown count: 1000 osd max scrubs: 1 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd mclock skip benchmark: true osd memory target: 1610612736 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub - Degraded data redundancy - is down - osds down - pg .*? is .*?degraded.*?, acting - pg .*? is stuck - pg degraded - PG_BACKFILL_FULL - Low space hindering backfill .*? backfill_toofull - OSD_ROOT_DOWN - pgs degraded - pgs undersized - is active.*backfill_toofull.* - \(POOL_APP_NOT_ENABLED\) mon_bind_msgr2: false sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 min_in: 4 workunit: branch: wip-yuri4-testing-2026-02-11-2126 sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.4 - osd.8 - osd.12 - - mon.b - osd.1 - osd.5 - osd.9 - osd.13 - - mon.c - osd.2 - osd.6 - osd.10 - osd.14 - - mgr.x - osd.3 - osd.7 - osd.11 - osd.15 - client.0 seed: 7172 sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri4-testing-2026-02-11-2126 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 targets: trial010.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAXPWfR+iNjeMutDmeGP9AEvhpvyIPKRC1q2WaqrgeJJPsIQp2xtayN84lbajZf9auB9m/uF4csHu3GJW69hSAs= trial060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK2yoY0CxFc0UNOg5WTSCqELVNtNa3WjSNY7pgG1VaRxtCUjjXONOdQdJixGekOz464VJ6yB45nQXan180suCNk= trial104.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI5ZCSKItpu018fsrT5rVMKPcjeA5QVHV13oZGHRWi+/L/Rfq4+vkTx+vgtqcp+HnI/P1LtX7WKEDHvxG6R0hDU= trial195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPGM+Bd5GNmzIkSGOuIvhEr7lqCirz7GJaCewJG8iQq2AZKCy42NRack/sjkWkI/rUUTXblq/JtI+H9i50vIpfg= tasks: - install: null - ceph: conf: osd: debug monc: 20 - exec: mon.a: - ceph config set mgr mgr/balancer/log_level debug - ceph balancer status - ceph osd set-require-min-compat-client reef - ceph balancer mode read - ceph balancer on - ceph balancer status - ceph balancer status detail - thrashosds: aggressive_pg_num_changes: false chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 min_in: 2 timeout: 1200 - rados: clients: - client.0 localize_reads: true objects: 50 op_weights: copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8bec0da71becad44414c54979f64c9ef0e7099c6 timestamp: 2026-02-12_15:38:14 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3932482 2026-02-12T17:20:19.672 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa; will attempt to use it 2026-02-12T17:20:19.673 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa/tasks 2026-02-12T17:20:19.673 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-12T17:20:19.675 INFO:teuthology.task.internal:Checking packages... 2026-02-12T17:20:19.675 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'afefaa40503a4bf024e96f97dd7f852b75f00ad6' 2026-02-12T17:20:19.675 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-12T17:20:19.675 INFO:teuthology.packaging:ref: None 2026-02-12T17:20:19.675 INFO:teuthology.packaging:tag: None 2026-02-12T17:20:19.675 INFO:teuthology.packaging:branch: wip-yuri4-testing-2026-02-11-2126 2026-02-12T17:20:19.675 INFO:teuthology.packaging:sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-12T17:20:19.675 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri4-testing-2026-02-11-2126 2026-02-12T17:20:19.794 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5230.gafefaa40 2026-02-12T17:20:19.795 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-12T17:20:19.806 INFO:teuthology.task.internal:no buildpackages task found 2026-02-12T17:20:19.806 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-12T17:20:19.820 INFO:teuthology.task.internal:Saving configuration 2026-02-12T17:20:19.835 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-12T17:20:19.851 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-12T17:20:19.931 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial010.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-12_15:38:14-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/46420', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-12 17:16:58.268216', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAXPWfR+iNjeMutDmeGP9AEvhpvyIPKRC1q2WaqrgeJJPsIQp2xtayN84lbajZf9auB9m/uF4csHu3GJW69hSAs='} 2026-02-12T17:20:19.987 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial060.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-12_15:38:14-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/46420', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-12 17:16:58.269055', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK2yoY0CxFc0UNOg5WTSCqELVNtNa3WjSNY7pgG1VaRxtCUjjXONOdQdJixGekOz464VJ6yB45nQXan180suCNk='} 2026-02-12T17:20:20.043 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial104.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-12_15:38:14-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/46420', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-12 17:16:58.268645', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI5ZCSKItpu018fsrT5rVMKPcjeA5QVHV13oZGHRWi+/L/Rfq4+vkTx+vgtqcp+HnI/P1LtX7WKEDHvxG6R0hDU='} 2026-02-12T17:20:20.097 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial195.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-12_15:38:14-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/46420', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-12 17:16:58.267523', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPGM+Bd5GNmzIkSGOuIvhEr7lqCirz7GJaCewJG8iQq2AZKCy42NRack/sjkWkI/rUUTXblq/JtI+H9i50vIpfg='} 2026-02-12T17:20:20.097 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-12T17:20:20.109 INFO:teuthology.task.internal:roles: ubuntu@trial010.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'] 2026-02-12T17:20:20.109 INFO:teuthology.task.internal:roles: ubuntu@trial060.front.sepia.ceph.com - ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'] 2026-02-12T17:20:20.109 INFO:teuthology.task.internal:roles: ubuntu@trial104.front.sepia.ceph.com - ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'] 2026-02-12T17:20:20.109 INFO:teuthology.task.internal:roles: ubuntu@trial195.front.sepia.ceph.com - ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0'] 2026-02-12T17:20:20.110 INFO:teuthology.run_tasks:Running task console_log... 2026-02-12T17:20:20.364 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x780c04ae7060>, signals=[15]) 2026-02-12T17:20:20.364 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-12T17:20:20.377 INFO:teuthology.task.internal:Opening connections... 2026-02-12T17:20:20.377 DEBUG:teuthology.task.internal:connecting to ubuntu@trial010.front.sepia.ceph.com 2026-02-12T17:20:20.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial010.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:20:20.441 DEBUG:teuthology.task.internal:connecting to ubuntu@trial060.front.sepia.ceph.com 2026-02-12T17:20:20.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:20:20.503 DEBUG:teuthology.task.internal:connecting to ubuntu@trial104.front.sepia.ceph.com 2026-02-12T17:20:20.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:20:20.561 DEBUG:teuthology.task.internal:connecting to ubuntu@trial195.front.sepia.ceph.com 2026-02-12T17:20:20.561 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:20:20.622 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-12T17:20:20.633 DEBUG:teuthology.orchestra.run.trial010:> uname -m 2026-02-12T17:20:20.649 INFO:teuthology.orchestra.run.trial010.stdout:x86_64 2026-02-12T17:20:20.649 DEBUG:teuthology.orchestra.run.trial010:> cat /etc/os-release 2026-02-12T17:20:20.705 INFO:teuthology.orchestra.run.trial010.stdout:NAME="CentOS Stream" 2026-02-12T17:20:20.705 INFO:teuthology.orchestra.run.trial010.stdout:VERSION="9" 2026-02-12T17:20:20.705 INFO:teuthology.orchestra.run.trial010.stdout:ID="centos" 2026-02-12T17:20:20.705 INFO:teuthology.orchestra.run.trial010.stdout:ID_LIKE="rhel fedora" 2026-02-12T17:20:20.705 INFO:teuthology.orchestra.run.trial010.stdout:VERSION_ID="9" 2026-02-12T17:20:20.705 INFO:teuthology.orchestra.run.trial010.stdout:PLATFORM_ID="platform:el9" 2026-02-12T17:20:20.705 INFO:teuthology.orchestra.run.trial010.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-12T17:20:20.705 INFO:teuthology.orchestra.run.trial010.stdout:ANSI_COLOR="0;31" 2026-02-12T17:20:20.705 INFO:teuthology.orchestra.run.trial010.stdout:LOGO="fedora-logo-icon" 2026-02-12T17:20:20.705 INFO:teuthology.orchestra.run.trial010.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-12T17:20:20.705 INFO:teuthology.orchestra.run.trial010.stdout:HOME_URL="https://centos.org/" 2026-02-12T17:20:20.705 INFO:teuthology.orchestra.run.trial010.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-12T17:20:20.705 INFO:teuthology.orchestra.run.trial010.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-12T17:20:20.705 INFO:teuthology.orchestra.run.trial010.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-12T17:20:20.706 INFO:teuthology.lock.ops:Updating trial010.front.sepia.ceph.com on lock server 2026-02-12T17:20:20.773 DEBUG:teuthology.orchestra.run.trial060:> uname -m 2026-02-12T17:20:20.788 INFO:teuthology.orchestra.run.trial060.stdout:x86_64 2026-02-12T17:20:20.788 DEBUG:teuthology.orchestra.run.trial060:> cat /etc/os-release 2026-02-12T17:20:20.845 INFO:teuthology.orchestra.run.trial060.stdout:NAME="CentOS Stream" 2026-02-12T17:20:20.845 INFO:teuthology.orchestra.run.trial060.stdout:VERSION="9" 2026-02-12T17:20:20.845 INFO:teuthology.orchestra.run.trial060.stdout:ID="centos" 2026-02-12T17:20:20.845 INFO:teuthology.orchestra.run.trial060.stdout:ID_LIKE="rhel fedora" 2026-02-12T17:20:20.845 INFO:teuthology.orchestra.run.trial060.stdout:VERSION_ID="9" 2026-02-12T17:20:20.845 INFO:teuthology.orchestra.run.trial060.stdout:PLATFORM_ID="platform:el9" 2026-02-12T17:20:20.845 INFO:teuthology.orchestra.run.trial060.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-12T17:20:20.845 INFO:teuthology.orchestra.run.trial060.stdout:ANSI_COLOR="0;31" 2026-02-12T17:20:20.845 INFO:teuthology.orchestra.run.trial060.stdout:LOGO="fedora-logo-icon" 2026-02-12T17:20:20.845 INFO:teuthology.orchestra.run.trial060.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-12T17:20:20.845 INFO:teuthology.orchestra.run.trial060.stdout:HOME_URL="https://centos.org/" 2026-02-12T17:20:20.845 INFO:teuthology.orchestra.run.trial060.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-12T17:20:20.845 INFO:teuthology.orchestra.run.trial060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-12T17:20:20.845 INFO:teuthology.orchestra.run.trial060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-12T17:20:20.846 INFO:teuthology.lock.ops:Updating trial060.front.sepia.ceph.com on lock server 2026-02-12T17:20:20.916 DEBUG:teuthology.orchestra.run.trial104:> uname -m 2026-02-12T17:20:20.931 INFO:teuthology.orchestra.run.trial104.stdout:x86_64 2026-02-12T17:20:20.931 DEBUG:teuthology.orchestra.run.trial104:> cat /etc/os-release 2026-02-12T17:20:20.987 INFO:teuthology.orchestra.run.trial104.stdout:NAME="CentOS Stream" 2026-02-12T17:20:20.988 INFO:teuthology.orchestra.run.trial104.stdout:VERSION="9" 2026-02-12T17:20:20.988 INFO:teuthology.orchestra.run.trial104.stdout:ID="centos" 2026-02-12T17:20:20.988 INFO:teuthology.orchestra.run.trial104.stdout:ID_LIKE="rhel fedora" 2026-02-12T17:20:20.988 INFO:teuthology.orchestra.run.trial104.stdout:VERSION_ID="9" 2026-02-12T17:20:20.988 INFO:teuthology.orchestra.run.trial104.stdout:PLATFORM_ID="platform:el9" 2026-02-12T17:20:20.988 INFO:teuthology.orchestra.run.trial104.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-12T17:20:20.988 INFO:teuthology.orchestra.run.trial104.stdout:ANSI_COLOR="0;31" 2026-02-12T17:20:20.988 INFO:teuthology.orchestra.run.trial104.stdout:LOGO="fedora-logo-icon" 2026-02-12T17:20:20.988 INFO:teuthology.orchestra.run.trial104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-12T17:20:20.988 INFO:teuthology.orchestra.run.trial104.stdout:HOME_URL="https://centos.org/" 2026-02-12T17:20:20.989 INFO:teuthology.orchestra.run.trial104.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-12T17:20:20.989 INFO:teuthology.orchestra.run.trial104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-12T17:20:20.989 INFO:teuthology.orchestra.run.trial104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-12T17:20:20.989 INFO:teuthology.lock.ops:Updating trial104.front.sepia.ceph.com on lock server 2026-02-12T17:20:21.071 DEBUG:teuthology.orchestra.run.trial195:> uname -m 2026-02-12T17:20:21.086 INFO:teuthology.orchestra.run.trial195.stdout:x86_64 2026-02-12T17:20:21.086 DEBUG:teuthology.orchestra.run.trial195:> cat /etc/os-release 2026-02-12T17:20:21.142 INFO:teuthology.orchestra.run.trial195.stdout:NAME="CentOS Stream" 2026-02-12T17:20:21.142 INFO:teuthology.orchestra.run.trial195.stdout:VERSION="9" 2026-02-12T17:20:21.142 INFO:teuthology.orchestra.run.trial195.stdout:ID="centos" 2026-02-12T17:20:21.142 INFO:teuthology.orchestra.run.trial195.stdout:ID_LIKE="rhel fedora" 2026-02-12T17:20:21.142 INFO:teuthology.orchestra.run.trial195.stdout:VERSION_ID="9" 2026-02-12T17:20:21.142 INFO:teuthology.orchestra.run.trial195.stdout:PLATFORM_ID="platform:el9" 2026-02-12T17:20:21.142 INFO:teuthology.orchestra.run.trial195.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-12T17:20:21.142 INFO:teuthology.orchestra.run.trial195.stdout:ANSI_COLOR="0;31" 2026-02-12T17:20:21.142 INFO:teuthology.orchestra.run.trial195.stdout:LOGO="fedora-logo-icon" 2026-02-12T17:20:21.142 INFO:teuthology.orchestra.run.trial195.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-12T17:20:21.142 INFO:teuthology.orchestra.run.trial195.stdout:HOME_URL="https://centos.org/" 2026-02-12T17:20:21.142 INFO:teuthology.orchestra.run.trial195.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-12T17:20:21.142 INFO:teuthology.orchestra.run.trial195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-12T17:20:21.142 INFO:teuthology.orchestra.run.trial195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-12T17:20:21.143 INFO:teuthology.lock.ops:Updating trial195.front.sepia.ceph.com on lock server 2026-02-12T17:20:21.200 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-12T17:20:21.214 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-12T17:20:21.225 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-12T17:20:21.225 DEBUG:teuthology.orchestra.run.trial010:> test '!' -e /home/ubuntu/cephtest 2026-02-12T17:20:21.227 DEBUG:teuthology.orchestra.run.trial060:> test '!' -e /home/ubuntu/cephtest 2026-02-12T17:20:21.228 DEBUG:teuthology.orchestra.run.trial104:> test '!' -e /home/ubuntu/cephtest 2026-02-12T17:20:21.229 DEBUG:teuthology.orchestra.run.trial195:> test '!' -e /home/ubuntu/cephtest 2026-02-12T17:20:21.245 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-12T17:20:21.260 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-12T17:20:21.260 DEBUG:teuthology.orchestra.run.trial010:> test -z $(ls -A /var/lib/ceph) 2026-02-12T17:20:21.282 DEBUG:teuthology.orchestra.run.trial060:> test -z $(ls -A /var/lib/ceph) 2026-02-12T17:20:21.284 DEBUG:teuthology.orchestra.run.trial104:> test -z $(ls -A /var/lib/ceph) 2026-02-12T17:20:21.285 DEBUG:teuthology.orchestra.run.trial195:> test -z $(ls -A /var/lib/ceph) 2026-02-12T17:20:21.302 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-12T17:20:21.568 INFO:teuthology.run_tasks:Running task kernel... 2026-02-12T17:20:21.588 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-12T17:20:21.588 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'mon.b': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'mon.c': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-12T17:20:21.588 DEBUG:teuthology.orchestra.run.trial010:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-12T17:20:21.588 DEBUG:teuthology.orchestra.run.trial060:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-12T17:20:21.588 DEBUG:teuthology.orchestra.run.trial104:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-12T17:20:21.589 DEBUG:teuthology.orchestra.run.trial195:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-12T17:20:21.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T17:20:21.603 DEBUG:teuthology.orchestra.run.trial010:> uname -r 2026-02-12T17:20:21.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T17:20:21.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T17:20:21.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T17:20:21.604 DEBUG:teuthology.orchestra.run.trial104:> uname -r 2026-02-12T17:20:21.604 DEBUG:teuthology.orchestra.run.trial060:> uname -r 2026-02-12T17:20:21.604 DEBUG:teuthology.orchestra.run.trial195:> uname -r 2026-02-12T17:20:21.659 INFO:teuthology.orchestra.run.trial060.stdout:5.14.0-645.el9.x86_64 2026-02-12T17:20:21.659 INFO:teuthology.task.kernel:Running kernel on trial060: 5.14.0-645.el9.x86_64 2026-02-12T17:20:21.659 DEBUG:teuthology.orchestra.run.trial060:> sudo yum install -y kernel 2026-02-12T17:20:21.660 INFO:teuthology.orchestra.run.trial104.stdout:5.14.0-645.el9.x86_64 2026-02-12T17:20:21.660 INFO:teuthology.orchestra.run.trial195.stdout:5.14.0-645.el9.x86_64 2026-02-12T17:20:21.660 INFO:teuthology.orchestra.run.trial010.stdout:5.14.0-645.el9.x86_64 2026-02-12T17:20:21.660 INFO:teuthology.task.kernel:Running kernel on trial104: 5.14.0-645.el9.x86_64 2026-02-12T17:20:21.661 DEBUG:teuthology.orchestra.run.trial104:> sudo yum install -y kernel 2026-02-12T17:20:21.661 INFO:teuthology.task.kernel:Running kernel on trial195: 5.14.0-645.el9.x86_64 2026-02-12T17:20:21.661 DEBUG:teuthology.orchestra.run.trial195:> sudo yum install -y kernel 2026-02-12T17:20:21.661 INFO:teuthology.task.kernel:Running kernel on trial010: 5.14.0-645.el9.x86_64 2026-02-12T17:20:21.661 DEBUG:teuthology.orchestra.run.trial010:> sudo yum install -y kernel 2026-02-12T17:20:22.898 INFO:teuthology.orchestra.run.trial104.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.9 MB 00:00 2026-02-12T17:20:23.547 INFO:teuthology.orchestra.run.trial010.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.9 MB 00:01 2026-02-12T17:20:23.616 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.9 MB 00:01 2026-02-12T17:20:23.747 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.9 MB 00:01 2026-02-12T17:20:26.553 INFO:teuthology.orchestra.run.trial104.stdout:CentOS Stream 9 - AppStream 8.6 MB/s | 27 MB 00:03 2026-02-12T17:20:26.758 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-12T17:20:27.086 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 27 MB 00:02 2026-02-12T17:20:28.792 INFO:teuthology.orchestra.run.trial010.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 27 MB 00:04 2026-02-12T17:20:29.995 INFO:teuthology.orchestra.run.trial104.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 7.8 MB 00:01 2026-02-12T17:20:30.111 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 7.8 MB 00:00 2026-02-12T17:20:30.838 INFO:teuthology.orchestra.run.trial104.stdout:CentOS Stream 9 - Extras packages 174 kB/s | 20 kB 00:00 2026-02-12T17:20:30.960 INFO:teuthology.orchestra.run.trial060.stdout:CentOS Stream 9 - Extras packages 150 kB/s | 20 kB 00:00 2026-02-12T17:20:31.751 INFO:teuthology.orchestra.run.trial104.stdout:Extra Packages for Enterprise Linux 24 MB/s | 20 MB 00:00 2026-02-12T17:20:32.016 INFO:teuthology.orchestra.run.trial010.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 7.8 MB 00:00 2026-02-12T17:20:32.316 INFO:teuthology.orchestra.run.trial060.stdout:Extra Packages for Enterprise Linux 16 MB/s | 20 MB 00:01 2026-02-12T17:20:35.015 INFO:teuthology.orchestra.run.trial104.stdout:lab-extras 2.1 MB/s | 50 kB 00:00 2026-02-12T17:20:35.208 INFO:teuthology.orchestra.run.trial010.stdout:CentOS Stream 9 - Extras packages 8.1 kB/s | 20 kB 00:02 2026-02-12T17:20:35.453 INFO:teuthology.orchestra.run.trial060.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-12T17:20:35.718 INFO:teuthology.orchestra.run.trial010.stdout:Extra Packages for Enterprise Linux 46 MB/s | 20 MB 00:00 2026-02-12T17:20:36.078 INFO:teuthology.orchestra.run.trial104.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-12T17:20:36.098 INFO:teuthology.orchestra.run.trial104.stdout:Dependencies resolved. 2026-02-12T17:20:36.099 INFO:teuthology.orchestra.run.trial104.stdout:================================================================================ 2026-02-12T17:20:36.099 INFO:teuthology.orchestra.run.trial104.stdout: Package Architecture Version Repository Size 2026-02-12T17:20:36.099 INFO:teuthology.orchestra.run.trial104.stdout:================================================================================ 2026-02-12T17:20:36.099 INFO:teuthology.orchestra.run.trial104.stdout:Installing: 2026-02-12T17:20:36.100 INFO:teuthology.orchestra.run.trial104.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-12T17:20:36.100 INFO:teuthology.orchestra.run.trial104.stdout:Installing dependencies: 2026-02-12T17:20:36.100 INFO:teuthology.orchestra.run.trial104.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-12T17:20:36.100 INFO:teuthology.orchestra.run.trial104.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-12T17:20:36.100 INFO:teuthology.orchestra.run.trial104.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-12T17:20:36.100 INFO:teuthology.orchestra.run.trial104.stdout: 2026-02-12T17:20:36.100 INFO:teuthology.orchestra.run.trial104.stdout:Transaction Summary 2026-02-12T17:20:36.100 INFO:teuthology.orchestra.run.trial104.stdout:================================================================================ 2026-02-12T17:20:36.100 INFO:teuthology.orchestra.run.trial104.stdout:Install 4 Packages 2026-02-12T17:20:36.100 INFO:teuthology.orchestra.run.trial104.stdout: 2026-02-12T17:20:36.100 INFO:teuthology.orchestra.run.trial104.stdout:Total download size: 88 M 2026-02-12T17:20:36.100 INFO:teuthology.orchestra.run.trial104.stdout:Installed size: 135 M 2026-02-12T17:20:36.101 INFO:teuthology.orchestra.run.trial104.stdout:Downloading Packages: 2026-02-12T17:20:36.517 INFO:teuthology.orchestra.run.trial060.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-12T17:20:36.537 INFO:teuthology.orchestra.run.trial060.stdout:Dependencies resolved. 2026-02-12T17:20:36.538 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-12T17:20:36.538 INFO:teuthology.orchestra.run.trial060.stdout: Package Architecture Version Repository Size 2026-02-12T17:20:36.538 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-12T17:20:36.538 INFO:teuthology.orchestra.run.trial060.stdout:Installing: 2026-02-12T17:20:36.538 INFO:teuthology.orchestra.run.trial060.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-12T17:20:36.538 INFO:teuthology.orchestra.run.trial060.stdout:Installing dependencies: 2026-02-12T17:20:36.538 INFO:teuthology.orchestra.run.trial060.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-12T17:20:36.538 INFO:teuthology.orchestra.run.trial060.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-12T17:20:36.539 INFO:teuthology.orchestra.run.trial060.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-12T17:20:36.539 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-12T17:20:36.539 INFO:teuthology.orchestra.run.trial060.stdout:Transaction Summary 2026-02-12T17:20:36.539 INFO:teuthology.orchestra.run.trial060.stdout:================================================================================ 2026-02-12T17:20:36.539 INFO:teuthology.orchestra.run.trial060.stdout:Install 4 Packages 2026-02-12T17:20:36.539 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-12T17:20:36.539 INFO:teuthology.orchestra.run.trial060.stdout:Total download size: 88 M 2026-02-12T17:20:36.539 INFO:teuthology.orchestra.run.trial060.stdout:Installed size: 135 M 2026-02-12T17:20:36.539 INFO:teuthology.orchestra.run.trial060.stdout:Downloading Packages: 2026-02-12T17:20:37.271 INFO:teuthology.orchestra.run.trial060.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 1.2 MB/s | 768 kB 00:00 2026-02-12T17:20:37.615 INFO:teuthology.orchestra.run.trial104.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 603 kB/s | 768 kB 00:01 2026-02-12T17:20:37.756 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - CRB 961 kB/s | 7.8 MB 00:08 2026-02-12T17:20:37.850 INFO:teuthology.orchestra.run.trial060.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 14 MB/s | 17 MB 00:01 2026-02-12T17:20:38.326 INFO:teuthology.orchestra.run.trial060.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 29 MB/s | 31 MB 00:01 2026-02-12T17:20:38.494 INFO:teuthology.orchestra.run.trial060.stdout:(4/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 21 MB/s | 40 MB 00:01 2026-02-12T17:20:38.494 INFO:teuthology.orchestra.run.trial060.stdout:-------------------------------------------------------------------------------- 2026-02-12T17:20:38.495 INFO:teuthology.orchestra.run.trial060.stdout:Total 45 MB/s | 88 MB 00:01 2026-02-12T17:20:38.686 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction check 2026-02-12T17:20:38.870 INFO:teuthology.orchestra.run.trial010.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-12T17:20:38.887 INFO:teuthology.orchestra.run.trial060.stdout:Transaction check succeeded. 2026-02-12T17:20:38.888 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction test 2026-02-12T17:20:39.233 INFO:teuthology.orchestra.run.trial060.stdout:Transaction test succeeded. 2026-02-12T17:20:39.234 INFO:teuthology.orchestra.run.trial060.stdout:Running transaction 2026-02-12T17:20:39.481 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 20 kB 00:01 2026-02-12T17:20:39.928 INFO:teuthology.orchestra.run.trial010.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-12T17:20:39.948 INFO:teuthology.orchestra.run.trial010.stdout:Dependencies resolved. 2026-02-12T17:20:39.949 INFO:teuthology.orchestra.run.trial010.stdout:================================================================================ 2026-02-12T17:20:39.949 INFO:teuthology.orchestra.run.trial010.stdout: Package Architecture Version Repository Size 2026-02-12T17:20:39.949 INFO:teuthology.orchestra.run.trial010.stdout:================================================================================ 2026-02-12T17:20:39.949 INFO:teuthology.orchestra.run.trial010.stdout:Installing: 2026-02-12T17:20:39.949 INFO:teuthology.orchestra.run.trial010.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-12T17:20:39.949 INFO:teuthology.orchestra.run.trial010.stdout:Installing dependencies: 2026-02-12T17:20:39.949 INFO:teuthology.orchestra.run.trial010.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-12T17:20:39.949 INFO:teuthology.orchestra.run.trial010.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-12T17:20:39.949 INFO:teuthology.orchestra.run.trial010.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-12T17:20:39.949 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T17:20:39.950 INFO:teuthology.orchestra.run.trial010.stdout:Transaction Summary 2026-02-12T17:20:39.950 INFO:teuthology.orchestra.run.trial010.stdout:================================================================================ 2026-02-12T17:20:39.950 INFO:teuthology.orchestra.run.trial010.stdout:Install 4 Packages 2026-02-12T17:20:39.950 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T17:20:39.950 INFO:teuthology.orchestra.run.trial010.stdout:Total download size: 88 M 2026-02-12T17:20:39.950 INFO:teuthology.orchestra.run.trial010.stdout:Installed size: 135 M 2026-02-12T17:20:39.950 INFO:teuthology.orchestra.run.trial010.stdout:Downloading Packages: 2026-02-12T17:20:40.324 INFO:teuthology.orchestra.run.trial060.stdout: Preparing : 1/1 2026-02-12T17:20:40.495 INFO:teuthology.orchestra.run.trial195.stdout:Extra Packages for Enterprise Linux 22 MB/s | 20 MB 00:00 2026-02-12T17:20:40.607 INFO:teuthology.orchestra.run.trial060.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-12T17:20:40.626 INFO:teuthology.orchestra.run.trial060.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T17:20:41.591 INFO:teuthology.orchestra.run.trial010.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 549 kB/s | 768 kB 00:01 2026-02-12T17:20:42.395 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T17:20:42.424 INFO:teuthology.orchestra.run.trial060.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T17:20:43.700 INFO:teuthology.orchestra.run.trial195.stdout:lab-extras 1.2 MB/s | 50 kB 00:00 2026-02-12T17:20:44.771 INFO:teuthology.orchestra.run.trial195.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-12T17:20:44.791 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-12T17:20:44.792 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-12T17:20:44.792 INFO:teuthology.orchestra.run.trial195.stdout: Package Architecture Version Repository Size 2026-02-12T17:20:44.792 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-12T17:20:44.792 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-12T17:20:44.792 INFO:teuthology.orchestra.run.trial195.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-12T17:20:44.792 INFO:teuthology.orchestra.run.trial195.stdout:Installing dependencies: 2026-02-12T17:20:44.792 INFO:teuthology.orchestra.run.trial195.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-12T17:20:44.792 INFO:teuthology.orchestra.run.trial195.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-12T17:20:44.792 INFO:teuthology.orchestra.run.trial195.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-12T17:20:44.793 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-12T17:20:44.793 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-12T17:20:44.793 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-12T17:20:44.793 INFO:teuthology.orchestra.run.trial195.stdout:Install 4 Packages 2026-02-12T17:20:44.793 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-12T17:20:44.793 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 88 M 2026-02-12T17:20:44.793 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 135 M 2026-02-12T17:20:44.793 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-12T17:20:45.451 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T17:20:45.458 INFO:teuthology.orchestra.run.trial060.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:20:45.535 INFO:teuthology.orchestra.run.trial195.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 1.2 MB/s | 768 kB 00:00 2026-02-12T17:20:46.090 INFO:teuthology.orchestra.run.trial195.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 14 MB/s | 17 MB 00:01 2026-02-12T17:20:46.617 INFO:teuthology.orchestra.run.trial195.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 28 MB/s | 31 MB 00:01 2026-02-12T17:20:46.774 INFO:teuthology.orchestra.run.trial195.stdout:(4/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 21 MB/s | 40 MB 00:01 2026-02-12T17:20:46.775 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-12T17:20:46.775 INFO:teuthology.orchestra.run.trial195.stdout:Total 44 MB/s | 88 MB 00:01 2026-02-12T17:20:46.965 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-12T17:20:47.168 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-12T17:20:47.168 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-12T17:20:47.513 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-12T17:20:47.514 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-12T17:20:48.553 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:20:48.601 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-12T17:20:48.885 INFO:teuthology.orchestra.run.trial195.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-12T17:20:48.904 INFO:teuthology.orchestra.run.trial195.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T17:20:50.677 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T17:20:50.707 INFO:teuthology.orchestra.run.trial195.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T17:20:53.728 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T17:20:53.735 INFO:teuthology.orchestra.run.trial195.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:20:55.674 INFO:teuthology.orchestra.run.trial104.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 903 kB/s | 17 MB 00:19 2026-02-12T17:20:56.766 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:21:00.332 INFO:teuthology.orchestra.run.trial010.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 867 kB/s | 17 MB 00:20 2026-02-12T17:21:00.617 INFO:teuthology.orchestra.run.trial104.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 1.3 MB/s | 31 MB 00:23 2026-02-12T17:21:16.054 INFO:teuthology.orchestra.run.trial010.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 915 kB/s | 31 MB 00:34 2026-02-12T17:21:19.140 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:21:19.164 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:21:19.334 INFO:teuthology.orchestra.run.trial060.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:21:19.334 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-12T17:21:19.334 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T17:21:19.334 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T17:21:19.387 INFO:teuthology.orchestra.run.trial060.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:21:19.387 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-12T17:21:19.387 INFO:teuthology.orchestra.run.trial060.stdout:Installed: 2026-02-12T17:21:19.387 INFO:teuthology.orchestra.run.trial060.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-12T17:21:19.387 INFO:teuthology.orchestra.run.trial060.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-12T17:21:19.387 INFO:teuthology.orchestra.run.trial060.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-12T17:21:19.387 INFO:teuthology.orchestra.run.trial060.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-12T17:21:19.387 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-12T17:21:19.387 INFO:teuthology.orchestra.run.trial060.stdout:Complete! 2026-02-12T17:21:19.445 DEBUG:teuthology.orchestra.run.trial060:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T17:21:19.469 INFO:teuthology.orchestra.run.trial060.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T17:21:19.470 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T17:21:19.470 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-12T17:21:19.470 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-12T17:21:19.470 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-12T17:21:19.470 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2026-02-12T17:21:19.470 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial060.front.sepia.ceph.com, path=None, version=distro) 2026-02-12T17:21:19.470 DEBUG:teuthology.orchestra.run.trial060:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T17:21:19.532 INFO:teuthology.orchestra.run.trial060.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T17:21:19.533 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T17:21:19.533 DEBUG:teuthology.orchestra.run.trial060:> sudo rpm -qi grub2-tools 2026-02-12T17:21:19.602 INFO:teuthology.orchestra.run.trial060.stdout:Name : grub2-tools 2026-02-12T17:21:19.602 INFO:teuthology.orchestra.run.trial060.stdout:Epoch : 1 2026-02-12T17:21:19.602 INFO:teuthology.orchestra.run.trial060.stdout:Version : 2.06 2026-02-12T17:21:19.602 INFO:teuthology.orchestra.run.trial060.stdout:Release : 118.el9 2026-02-12T17:21:19.602 INFO:teuthology.orchestra.run.trial060.stdout:Architecture: x86_64 2026-02-12T17:21:19.603 INFO:teuthology.orchestra.run.trial060.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-12T17:21:19.603 INFO:teuthology.orchestra.run.trial060.stdout:Group : Unspecified 2026-02-12T17:21:19.603 INFO:teuthology.orchestra.run.trial060.stdout:Size : 8393909 2026-02-12T17:21:19.603 INFO:teuthology.orchestra.run.trial060.stdout:License : GPLv3+ 2026-02-12T17:21:19.603 INFO:teuthology.orchestra.run.trial060.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-12T17:21:19.603 INFO:teuthology.orchestra.run.trial060.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-12T17:21:19.603 INFO:teuthology.orchestra.run.trial060.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-12T17:21:19.603 INFO:teuthology.orchestra.run.trial060.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-12T17:21:19.603 INFO:teuthology.orchestra.run.trial060.stdout:Packager : builder@centos.org 2026-02-12T17:21:19.603 INFO:teuthology.orchestra.run.trial060.stdout:Vendor : CentOS 2026-02-12T17:21:19.603 INFO:teuthology.orchestra.run.trial060.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-12T17:21:19.604 INFO:teuthology.orchestra.run.trial060.stdout:Summary : Support tools for GRUB. 2026-02-12T17:21:19.604 INFO:teuthology.orchestra.run.trial060.stdout:Description : 2026-02-12T17:21:19.604 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-12T17:21:19.604 INFO:teuthology.orchestra.run.trial060.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-12T17:21:19.604 INFO:teuthology.orchestra.run.trial060.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-12T17:21:19.604 INFO:teuthology.orchestra.run.trial060.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-12T17:21:19.604 INFO:teuthology.orchestra.run.trial060.stdout:hardware devices. 2026-02-12T17:21:19.604 INFO:teuthology.orchestra.run.trial060.stdout: 2026-02-12T17:21:19.604 INFO:teuthology.orchestra.run.trial060.stdout:This subpackage provides tools for support of all platforms. 2026-02-12T17:21:19.605 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-12T17:21:19.606 INFO:teuthology.task.kernel:Updating grub on trial060 to boot 5.14.0-677.el9.x86_64 2026-02-12T17:21:19.606 DEBUG:teuthology.orchestra.run.trial060:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-12T17:21:19.671 INFO:teuthology.orchestra.run.trial060.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-12T17:21:19.672 DEBUG:teuthology.orchestra.run.trial060:> sudo command -v grubby && echo yes || echo no 2026-02-12T17:21:19.736 INFO:teuthology.orchestra.run.trial060.stdout:/sbin/grubby 2026-02-12T17:21:19.737 INFO:teuthology.orchestra.run.trial060.stdout:yes 2026-02-12T17:21:19.738 DEBUG:teuthology.orchestra.run.trial060:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-12T17:21:19.801 INFO:teuthology.orchestra.run.trial060.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-12T17:21:19.802 DEBUG:teuthology.orchestra.run.trial060:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-12T17:21:19.874 DEBUG:teuthology.orchestra.run.trial060:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-12T17:21:20.082 INFO:teuthology.orchestra.run.trial060.stderr:Generating grub configuration file ... 2026-02-12T17:21:20.147 INFO:teuthology.orchestra.run.trial060.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-12T17:21:20.153 INFO:teuthology.orchestra.run.trial060.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-12T17:21:20.402 INFO:teuthology.orchestra.run.trial060.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-12T17:21:20.408 INFO:teuthology.orchestra.run.trial060.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-12T17:21:20.440 INFO:teuthology.orchestra.run.trial060.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-12T17:21:20.446 INFO:teuthology.orchestra.run.trial060.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-12T17:21:20.469 INFO:teuthology.orchestra.run.trial060.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-12T17:21:20.476 INFO:teuthology.orchestra.run.trial060.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-12T17:21:20.530 INFO:teuthology.orchestra.run.trial104.stdout:(4/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 918 kB/s | 40 MB 00:44 2026-02-12T17:21:20.532 INFO:teuthology.orchestra.run.trial104.stdout:-------------------------------------------------------------------------------- 2026-02-12T17:21:20.532 INFO:teuthology.orchestra.run.trial104.stdout:Total 2.0 MB/s | 88 MB 00:44 2026-02-12T17:21:20.557 INFO:teuthology.orchestra.run.trial060.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-12T17:21:20.565 INFO:teuthology.orchestra.run.trial060.stderr:done 2026-02-12T17:21:20.567 DEBUG:teuthology.orchestra.run.trial060:> test -d /sys/firmware/efi 2026-02-12T17:21:20.583 DEBUG:teuthology.orchestra.run.trial060:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-12T17:21:20.648 INFO:teuthology.orchestra.run.trial060.stdout:/boot/efi/EFI/BOOT 2026-02-12T17:21:20.649 INFO:teuthology.orchestra.run.trial060.stdout:/boot/efi/EFI/centos 2026-02-12T17:21:20.650 DEBUG:teuthology.orchestra.run.trial060:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-12T17:21:20.716 INFO:teuthology.orchestra.run.trial060.stdout:no 2026-02-12T17:21:20.717 DEBUG:teuthology.orchestra.run.trial060:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-12T17:21:20.718 INFO:teuthology.orchestra.run.trial104.stdout:Running transaction check 2026-02-12T17:21:20.780 INFO:teuthology.orchestra.run.trial060.stdout:yes 2026-02-12T17:21:20.781 DEBUG:teuthology.orchestra.run.trial060:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-12T17:21:20.846 DEBUG:teuthology.orchestra.run.trial060:> sudo shutdown -r now 2026-02-12T17:21:20.919 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2026-02-12T17:21:20.928 INFO:teuthology.orchestra.run.trial104.stdout:Transaction check succeeded. 2026-02-12T17:21:20.928 INFO:teuthology.orchestra.run.trial104.stdout:Running transaction test 2026-02-12T17:21:21.274 INFO:teuthology.orchestra.run.trial104.stdout:Transaction test succeeded. 2026-02-12T17:21:21.274 INFO:teuthology.orchestra.run.trial104.stdout:Running transaction 2026-02-12T17:21:22.375 INFO:teuthology.orchestra.run.trial104.stdout: Preparing : 1/1 2026-02-12T17:21:22.663 INFO:teuthology.orchestra.run.trial104.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-12T17:21:22.682 INFO:teuthology.orchestra.run.trial104.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T17:21:24.458 INFO:teuthology.orchestra.run.trial104.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T17:21:24.487 INFO:teuthology.orchestra.run.trial104.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T17:21:24.696 INFO:teuthology.orchestra.run.trial010.stdout:(4/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 911 kB/s | 40 MB 00:44 2026-02-12T17:21:24.697 INFO:teuthology.orchestra.run.trial010.stdout:-------------------------------------------------------------------------------- 2026-02-12T17:21:24.698 INFO:teuthology.orchestra.run.trial010.stdout:Total 2.0 MB/s | 88 MB 00:44 2026-02-12T17:21:24.885 INFO:teuthology.orchestra.run.trial010.stdout:Running transaction check 2026-02-12T17:21:25.088 INFO:teuthology.orchestra.run.trial010.stdout:Transaction check succeeded. 2026-02-12T17:21:25.088 INFO:teuthology.orchestra.run.trial010.stdout:Running transaction test 2026-02-12T17:21:25.434 INFO:teuthology.orchestra.run.trial010.stdout:Transaction test succeeded. 2026-02-12T17:21:25.434 INFO:teuthology.orchestra.run.trial010.stdout:Running transaction 2026-02-12T17:21:26.536 INFO:teuthology.orchestra.run.trial010.stdout: Preparing : 1/1 2026-02-12T17:21:26.825 INFO:teuthology.orchestra.run.trial010.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-12T17:21:26.844 INFO:teuthology.orchestra.run.trial010.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T17:21:27.516 INFO:teuthology.orchestra.run.trial104.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T17:21:27.522 INFO:teuthology.orchestra.run.trial104.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:21:27.552 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:21:27.574 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:21:27.745 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:21:27.745 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-12T17:21:27.745 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T17:21:27.745 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T17:21:27.797 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:21:27.797 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-12T17:21:27.797 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-12T17:21:27.797 INFO:teuthology.orchestra.run.trial195.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-12T17:21:27.797 INFO:teuthology.orchestra.run.trial195.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-12T17:21:27.797 INFO:teuthology.orchestra.run.trial195.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-12T17:21:27.797 INFO:teuthology.orchestra.run.trial195.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-12T17:21:27.797 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-12T17:21:27.797 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-12T17:21:27.853 DEBUG:teuthology.orchestra.run.trial195:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T17:21:27.878 INFO:teuthology.orchestra.run.trial195.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T17:21:27.878 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T17:21:27.878 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-12T17:21:27.878 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-12T17:21:27.878 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-12T17:21:27.878 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2026-02-12T17:21:27.878 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial195.front.sepia.ceph.com, path=None, version=distro) 2026-02-12T17:21:27.878 DEBUG:teuthology.orchestra.run.trial195:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T17:21:27.942 INFO:teuthology.orchestra.run.trial195.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T17:21:27.943 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T17:21:27.943 DEBUG:teuthology.orchestra.run.trial195:> sudo rpm -qi grub2-tools 2026-02-12T17:21:28.013 INFO:teuthology.orchestra.run.trial195.stdout:Name : grub2-tools 2026-02-12T17:21:28.014 INFO:teuthology.orchestra.run.trial195.stdout:Epoch : 1 2026-02-12T17:21:28.014 INFO:teuthology.orchestra.run.trial195.stdout:Version : 2.06 2026-02-12T17:21:28.014 INFO:teuthology.orchestra.run.trial195.stdout:Release : 118.el9 2026-02-12T17:21:28.014 INFO:teuthology.orchestra.run.trial195.stdout:Architecture: x86_64 2026-02-12T17:21:28.014 INFO:teuthology.orchestra.run.trial195.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-12T17:21:28.014 INFO:teuthology.orchestra.run.trial195.stdout:Group : Unspecified 2026-02-12T17:21:28.014 INFO:teuthology.orchestra.run.trial195.stdout:Size : 8393909 2026-02-12T17:21:28.014 INFO:teuthology.orchestra.run.trial195.stdout:License : GPLv3+ 2026-02-12T17:21:28.014 INFO:teuthology.orchestra.run.trial195.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-12T17:21:28.014 INFO:teuthology.orchestra.run.trial195.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-12T17:21:28.014 INFO:teuthology.orchestra.run.trial195.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-12T17:21:28.015 INFO:teuthology.orchestra.run.trial195.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-12T17:21:28.015 INFO:teuthology.orchestra.run.trial195.stdout:Packager : builder@centos.org 2026-02-12T17:21:28.015 INFO:teuthology.orchestra.run.trial195.stdout:Vendor : CentOS 2026-02-12T17:21:28.015 INFO:teuthology.orchestra.run.trial195.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-12T17:21:28.015 INFO:teuthology.orchestra.run.trial195.stdout:Summary : Support tools for GRUB. 2026-02-12T17:21:28.015 INFO:teuthology.orchestra.run.trial195.stdout:Description : 2026-02-12T17:21:28.015 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-12T17:21:28.015 INFO:teuthology.orchestra.run.trial195.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-12T17:21:28.015 INFO:teuthology.orchestra.run.trial195.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-12T17:21:28.015 INFO:teuthology.orchestra.run.trial195.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-12T17:21:28.015 INFO:teuthology.orchestra.run.trial195.stdout:hardware devices. 2026-02-12T17:21:28.016 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-12T17:21:28.016 INFO:teuthology.orchestra.run.trial195.stdout:This subpackage provides tools for support of all platforms. 2026-02-12T17:21:28.017 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-12T17:21:28.017 INFO:teuthology.task.kernel:Updating grub on trial195 to boot 5.14.0-677.el9.x86_64 2026-02-12T17:21:28.017 DEBUG:teuthology.orchestra.run.trial195:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-12T17:21:28.081 INFO:teuthology.orchestra.run.trial195.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-12T17:21:28.082 DEBUG:teuthology.orchestra.run.trial195:> sudo command -v grubby && echo yes || echo no 2026-02-12T17:21:28.146 INFO:teuthology.orchestra.run.trial195.stdout:/sbin/grubby 2026-02-12T17:21:28.147 INFO:teuthology.orchestra.run.trial195.stdout:yes 2026-02-12T17:21:28.148 DEBUG:teuthology.orchestra.run.trial195:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-12T17:21:28.212 INFO:teuthology.orchestra.run.trial195.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-12T17:21:28.213 DEBUG:teuthology.orchestra.run.trial195:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-12T17:21:28.287 DEBUG:teuthology.orchestra.run.trial195:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-12T17:21:28.499 INFO:teuthology.orchestra.run.trial195.stderr:Generating grub configuration file ... 2026-02-12T17:21:28.563 INFO:teuthology.orchestra.run.trial195.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-12T17:21:28.570 INFO:teuthology.orchestra.run.trial195.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-12T17:21:28.610 INFO:teuthology.orchestra.run.trial010.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T17:21:28.639 INFO:teuthology.orchestra.run.trial010.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T17:21:28.825 INFO:teuthology.orchestra.run.trial195.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-12T17:21:28.831 INFO:teuthology.orchestra.run.trial195.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-12T17:21:28.863 INFO:teuthology.orchestra.run.trial195.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-12T17:21:28.870 INFO:teuthology.orchestra.run.trial195.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-12T17:21:28.893 INFO:teuthology.orchestra.run.trial195.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-12T17:21:28.900 INFO:teuthology.orchestra.run.trial195.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-12T17:21:28.983 INFO:teuthology.orchestra.run.trial195.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-12T17:21:28.991 INFO:teuthology.orchestra.run.trial195.stderr:done 2026-02-12T17:21:28.993 DEBUG:teuthology.orchestra.run.trial195:> test -d /sys/firmware/efi 2026-02-12T17:21:29.008 DEBUG:teuthology.orchestra.run.trial195:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-12T17:21:29.073 INFO:teuthology.orchestra.run.trial195.stdout:/boot/efi/EFI/BOOT 2026-02-12T17:21:29.073 INFO:teuthology.orchestra.run.trial195.stdout:/boot/efi/EFI/centos 2026-02-12T17:21:29.074 DEBUG:teuthology.orchestra.run.trial195:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-12T17:21:29.139 INFO:teuthology.orchestra.run.trial195.stdout:no 2026-02-12T17:21:29.140 DEBUG:teuthology.orchestra.run.trial195:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-12T17:21:29.205 INFO:teuthology.orchestra.run.trial195.stdout:yes 2026-02-12T17:21:29.205 DEBUG:teuthology.orchestra.run.trial195:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-12T17:21:29.269 DEBUG:teuthology.orchestra.run.trial195:> sudo shutdown -r now 2026-02-12T17:21:30.779 INFO:teuthology.orchestra.run.trial104.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:21:31.883 INFO:teuthology.orchestra.run.trial010.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T17:21:31.889 INFO:teuthology.orchestra.run.trial010.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:21:34.923 INFO:teuthology.orchestra.run.trial010.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:21:50.888 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2026-02-12T17:21:50.888 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial060.front.sepia.ceph.com' 2026-02-12T17:21:50.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:21:59.312 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2026-02-12T17:21:59.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial195.front.sepia.ceph.com' 2026-02-12T17:21:59.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:22:01.218 INFO:teuthology.orchestra.run.trial104.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:22:01.238 INFO:teuthology.orchestra.run.trial104.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:22:01.300 INFO:teuthology.orchestra.run.trial104.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:22:01.300 INFO:teuthology.orchestra.run.trial104.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-12T17:22:01.301 INFO:teuthology.orchestra.run.trial104.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T17:22:01.301 INFO:teuthology.orchestra.run.trial104.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T17:22:01.352 INFO:teuthology.orchestra.run.trial104.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:22:01.352 INFO:teuthology.orchestra.run.trial104.stdout: 2026-02-12T17:22:01.352 INFO:teuthology.orchestra.run.trial104.stdout:Installed: 2026-02-12T17:22:01.352 INFO:teuthology.orchestra.run.trial104.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-12T17:22:01.352 INFO:teuthology.orchestra.run.trial104.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-12T17:22:01.352 INFO:teuthology.orchestra.run.trial104.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-12T17:22:01.352 INFO:teuthology.orchestra.run.trial104.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-12T17:22:01.352 INFO:teuthology.orchestra.run.trial104.stdout: 2026-02-12T17:22:01.352 INFO:teuthology.orchestra.run.trial104.stdout:Complete! 2026-02-12T17:22:01.409 DEBUG:teuthology.orchestra.run.trial104:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T17:22:01.435 INFO:teuthology.orchestra.run.trial104.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T17:22:01.436 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T17:22:01.436 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-12T17:22:01.436 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-12T17:22:01.436 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-12T17:22:01.437 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2026-02-12T17:22:01.437 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial104.front.sepia.ceph.com, path=None, version=distro) 2026-02-12T17:22:01.437 DEBUG:teuthology.orchestra.run.trial104:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T17:22:01.500 INFO:teuthology.orchestra.run.trial104.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T17:22:01.500 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T17:22:01.500 DEBUG:teuthology.orchestra.run.trial104:> sudo rpm -qi grub2-tools 2026-02-12T17:22:01.569 INFO:teuthology.orchestra.run.trial104.stdout:Name : grub2-tools 2026-02-12T17:22:01.569 INFO:teuthology.orchestra.run.trial104.stdout:Epoch : 1 2026-02-12T17:22:01.569 INFO:teuthology.orchestra.run.trial104.stdout:Version : 2.06 2026-02-12T17:22:01.569 INFO:teuthology.orchestra.run.trial104.stdout:Release : 118.el9 2026-02-12T17:22:01.570 INFO:teuthology.orchestra.run.trial104.stdout:Architecture: x86_64 2026-02-12T17:22:01.570 INFO:teuthology.orchestra.run.trial104.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-12T17:22:01.570 INFO:teuthology.orchestra.run.trial104.stdout:Group : Unspecified 2026-02-12T17:22:01.570 INFO:teuthology.orchestra.run.trial104.stdout:Size : 8393909 2026-02-12T17:22:01.570 INFO:teuthology.orchestra.run.trial104.stdout:License : GPLv3+ 2026-02-12T17:22:01.570 INFO:teuthology.orchestra.run.trial104.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-12T17:22:01.570 INFO:teuthology.orchestra.run.trial104.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-12T17:22:01.570 INFO:teuthology.orchestra.run.trial104.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-12T17:22:01.570 INFO:teuthology.orchestra.run.trial104.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-12T17:22:01.570 INFO:teuthology.orchestra.run.trial104.stdout:Packager : builder@centos.org 2026-02-12T17:22:01.570 INFO:teuthology.orchestra.run.trial104.stdout:Vendor : CentOS 2026-02-12T17:22:01.570 INFO:teuthology.orchestra.run.trial104.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-12T17:22:01.571 INFO:teuthology.orchestra.run.trial104.stdout:Summary : Support tools for GRUB. 2026-02-12T17:22:01.571 INFO:teuthology.orchestra.run.trial104.stdout:Description : 2026-02-12T17:22:01.571 INFO:teuthology.orchestra.run.trial104.stdout: 2026-02-12T17:22:01.571 INFO:teuthology.orchestra.run.trial104.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-12T17:22:01.571 INFO:teuthology.orchestra.run.trial104.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-12T17:22:01.571 INFO:teuthology.orchestra.run.trial104.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-12T17:22:01.571 INFO:teuthology.orchestra.run.trial104.stdout:hardware devices. 2026-02-12T17:22:01.571 INFO:teuthology.orchestra.run.trial104.stdout: 2026-02-12T17:22:01.571 INFO:teuthology.orchestra.run.trial104.stdout:This subpackage provides tools for support of all platforms. 2026-02-12T17:22:01.572 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-12T17:22:01.573 INFO:teuthology.task.kernel:Updating grub on trial104 to boot 5.14.0-677.el9.x86_64 2026-02-12T17:22:01.573 DEBUG:teuthology.orchestra.run.trial104:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-12T17:22:01.635 INFO:teuthology.orchestra.run.trial104.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-12T17:22:01.637 DEBUG:teuthology.orchestra.run.trial104:> sudo command -v grubby && echo yes || echo no 2026-02-12T17:22:01.700 INFO:teuthology.orchestra.run.trial104.stdout:/sbin/grubby 2026-02-12T17:22:01.702 INFO:teuthology.orchestra.run.trial104.stdout:yes 2026-02-12T17:22:01.702 DEBUG:teuthology.orchestra.run.trial104:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-12T17:22:01.766 INFO:teuthology.orchestra.run.trial104.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-12T17:22:01.767 DEBUG:teuthology.orchestra.run.trial104:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-12T17:22:01.840 DEBUG:teuthology.orchestra.run.trial104:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-12T17:22:02.048 INFO:teuthology.orchestra.run.trial104.stderr:Generating grub configuration file ... 2026-02-12T17:22:02.110 INFO:teuthology.orchestra.run.trial104.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-12T17:22:02.117 INFO:teuthology.orchestra.run.trial104.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-12T17:22:02.366 INFO:teuthology.orchestra.run.trial104.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-12T17:22:02.373 INFO:teuthology.orchestra.run.trial104.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-12T17:22:02.405 INFO:teuthology.orchestra.run.trial104.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-12T17:22:02.412 INFO:teuthology.orchestra.run.trial104.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-12T17:22:02.435 INFO:teuthology.orchestra.run.trial104.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-12T17:22:02.441 INFO:teuthology.orchestra.run.trial104.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-12T17:22:02.523 INFO:teuthology.orchestra.run.trial104.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-12T17:22:02.531 INFO:teuthology.orchestra.run.trial104.stderr:done 2026-02-12T17:22:02.533 DEBUG:teuthology.orchestra.run.trial104:> test -d /sys/firmware/efi 2026-02-12T17:22:02.549 DEBUG:teuthology.orchestra.run.trial104:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-12T17:22:02.614 INFO:teuthology.orchestra.run.trial104.stdout:/boot/efi/EFI/BOOT 2026-02-12T17:22:02.614 INFO:teuthology.orchestra.run.trial104.stdout:/boot/efi/EFI/centos 2026-02-12T17:22:02.615 DEBUG:teuthology.orchestra.run.trial104:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-12T17:22:02.681 INFO:teuthology.orchestra.run.trial104.stdout:no 2026-02-12T17:22:02.682 DEBUG:teuthology.orchestra.run.trial104:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-12T17:22:02.745 INFO:teuthology.orchestra.run.trial104.stdout:yes 2026-02-12T17:22:02.746 DEBUG:teuthology.orchestra.run.trial104:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-12T17:22:02.809 DEBUG:teuthology.orchestra.run.trial104:> sudo shutdown -r now 2026-02-12T17:22:02.882 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2026-02-12T17:22:05.227 INFO:teuthology.orchestra.run.trial010.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:22:05.248 INFO:teuthology.orchestra.run.trial010.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:22:05.310 INFO:teuthology.orchestra.run.trial010.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:22:05.310 INFO:teuthology.orchestra.run.trial010.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-12T17:22:05.310 INFO:teuthology.orchestra.run.trial010.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T17:22:05.310 INFO:teuthology.orchestra.run.trial010.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T17:22:05.362 INFO:teuthology.orchestra.run.trial010.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T17:22:05.362 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T17:22:05.362 INFO:teuthology.orchestra.run.trial010.stdout:Installed: 2026-02-12T17:22:05.362 INFO:teuthology.orchestra.run.trial010.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-12T17:22:05.362 INFO:teuthology.orchestra.run.trial010.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-12T17:22:05.362 INFO:teuthology.orchestra.run.trial010.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-12T17:22:05.362 INFO:teuthology.orchestra.run.trial010.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-12T17:22:05.362 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T17:22:05.362 INFO:teuthology.orchestra.run.trial010.stdout:Complete! 2026-02-12T17:22:05.419 DEBUG:teuthology.orchestra.run.trial010:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T17:22:05.443 INFO:teuthology.orchestra.run.trial010.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T17:22:05.444 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T17:22:05.444 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-12T17:22:05.444 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-12T17:22:05.444 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-12T17:22:05.444 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-12T17:22:05.444 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial010.front.sepia.ceph.com, path=None, version=distro) 2026-02-12T17:22:05.444 DEBUG:teuthology.orchestra.run.trial010:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T17:22:05.507 INFO:teuthology.orchestra.run.trial010.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T17:22:05.508 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T17:22:05.508 DEBUG:teuthology.orchestra.run.trial010:> sudo rpm -qi grub2-tools 2026-02-12T17:22:05.577 INFO:teuthology.orchestra.run.trial010.stdout:Name : grub2-tools 2026-02-12T17:22:05.577 INFO:teuthology.orchestra.run.trial010.stdout:Epoch : 1 2026-02-12T17:22:05.578 INFO:teuthology.orchestra.run.trial010.stdout:Version : 2.06 2026-02-12T17:22:05.578 INFO:teuthology.orchestra.run.trial010.stdout:Release : 118.el9 2026-02-12T17:22:05.578 INFO:teuthology.orchestra.run.trial010.stdout:Architecture: x86_64 2026-02-12T17:22:05.578 INFO:teuthology.orchestra.run.trial010.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-12T17:22:05.578 INFO:teuthology.orchestra.run.trial010.stdout:Group : Unspecified 2026-02-12T17:22:05.578 INFO:teuthology.orchestra.run.trial010.stdout:Size : 8393909 2026-02-12T17:22:05.578 INFO:teuthology.orchestra.run.trial010.stdout:License : GPLv3+ 2026-02-12T17:22:05.578 INFO:teuthology.orchestra.run.trial010.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-12T17:22:05.578 INFO:teuthology.orchestra.run.trial010.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-12T17:22:05.578 INFO:teuthology.orchestra.run.trial010.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-12T17:22:05.579 INFO:teuthology.orchestra.run.trial010.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-12T17:22:05.579 INFO:teuthology.orchestra.run.trial010.stdout:Packager : builder@centos.org 2026-02-12T17:22:05.579 INFO:teuthology.orchestra.run.trial010.stdout:Vendor : CentOS 2026-02-12T17:22:05.579 INFO:teuthology.orchestra.run.trial010.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-12T17:22:05.579 INFO:teuthology.orchestra.run.trial010.stdout:Summary : Support tools for GRUB. 2026-02-12T17:22:05.579 INFO:teuthology.orchestra.run.trial010.stdout:Description : 2026-02-12T17:22:05.579 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T17:22:05.579 INFO:teuthology.orchestra.run.trial010.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-12T17:22:05.579 INFO:teuthology.orchestra.run.trial010.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-12T17:22:05.579 INFO:teuthology.orchestra.run.trial010.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-12T17:22:05.579 INFO:teuthology.orchestra.run.trial010.stdout:hardware devices. 2026-02-12T17:22:05.579 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T17:22:05.580 INFO:teuthology.orchestra.run.trial010.stdout:This subpackage provides tools for support of all platforms. 2026-02-12T17:22:05.581 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-12T17:22:05.581 INFO:teuthology.task.kernel:Updating grub on trial010 to boot 5.14.0-677.el9.x86_64 2026-02-12T17:22:05.581 DEBUG:teuthology.orchestra.run.trial010:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-12T17:22:05.646 INFO:teuthology.orchestra.run.trial010.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-12T17:22:05.647 DEBUG:teuthology.orchestra.run.trial010:> sudo command -v grubby && echo yes || echo no 2026-02-12T17:22:05.710 INFO:teuthology.orchestra.run.trial010.stdout:/sbin/grubby 2026-02-12T17:22:05.711 INFO:teuthology.orchestra.run.trial010.stdout:yes 2026-02-12T17:22:05.711 DEBUG:teuthology.orchestra.run.trial010:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-12T17:22:05.775 INFO:teuthology.orchestra.run.trial010.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-12T17:22:05.776 DEBUG:teuthology.orchestra.run.trial010:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-12T17:22:05.849 DEBUG:teuthology.orchestra.run.trial010:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-12T17:22:06.056 INFO:teuthology.orchestra.run.trial010.stderr:Generating grub configuration file ... 2026-02-12T17:22:06.119 INFO:teuthology.orchestra.run.trial010.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-12T17:22:06.126 INFO:teuthology.orchestra.run.trial010.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-12T17:22:06.373 INFO:teuthology.orchestra.run.trial010.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-12T17:22:06.380 INFO:teuthology.orchestra.run.trial010.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-12T17:22:06.411 INFO:teuthology.orchestra.run.trial010.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-12T17:22:06.418 INFO:teuthology.orchestra.run.trial010.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-12T17:22:06.441 INFO:teuthology.orchestra.run.trial010.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-12T17:22:06.447 INFO:teuthology.orchestra.run.trial010.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-12T17:22:06.530 INFO:teuthology.orchestra.run.trial010.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-12T17:22:06.538 INFO:teuthology.orchestra.run.trial010.stderr:done 2026-02-12T17:22:06.540 DEBUG:teuthology.orchestra.run.trial010:> test -d /sys/firmware/efi 2026-02-12T17:22:06.556 DEBUG:teuthology.orchestra.run.trial010:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-12T17:22:06.623 INFO:teuthology.orchestra.run.trial010.stdout:/boot/efi/EFI/BOOT 2026-02-12T17:22:06.623 INFO:teuthology.orchestra.run.trial010.stdout:/boot/efi/EFI/centos 2026-02-12T17:22:06.625 DEBUG:teuthology.orchestra.run.trial010:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-12T17:22:06.690 INFO:teuthology.orchestra.run.trial010.stdout:no 2026-02-12T17:22:06.690 DEBUG:teuthology.orchestra.run.trial010:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-12T17:22:06.755 INFO:teuthology.orchestra.run.trial010.stdout:yes 2026-02-12T17:22:06.756 DEBUG:teuthology.orchestra.run.trial010:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-12T17:22:06.819 DEBUG:teuthology.orchestra.run.trial010:> sudo shutdown -r now 2026-02-12T17:22:06.894 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2026-02-12T17:22:32.878 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2026-02-12T17:22:32.878 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:22:32.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:22:36.866 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-12T17:22:36.867 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial010.front.sepia.ceph.com' 2026-02-12T17:22:36.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial010.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:22:47.133 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:22:50.895 DEBUG:teuthology.orchestra.remote:timed out 2026-02-12T17:22:56.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:22:56.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:22:59.229 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:22:59.315 DEBUG:teuthology.orchestra.remote:timed out 2026-02-12T17:22:59.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial060.front.sepia.ceph.com' 2026-02-12T17:22:59.897 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:23:00.031 DEBUG:teuthology.orchestra.run.trial060:> true 2026-02-12T17:23:00.192 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial060.front.sepia.ceph.com' 2026-02-12T17:23:00.192 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-677.el9.x86_64"... 2026-02-12T17:23:00.192 DEBUG:teuthology.orchestra.run.trial060:> uname -r 2026-02-12T17:23:00.247 INFO:teuthology.orchestra.run.trial060.stdout:5.14.0-677.el9.x86_64 2026-02-12T17:23:00.248 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-12T17:23:00.248 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-12T17:23:00.248 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-12T17:23:01.249 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2026-02-12T17:23:01.249 DEBUG:teuthology.orchestra.run.trial060:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-12T17:23:01.278 INFO:teuthology.orchestra.run.trial060.stdout:ttyS1 2026-02-12T17:23:01.304 DEBUG:teuthology.parallel:result is None 2026-02-12T17:23:08.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial195.front.sepia.ceph.com' 2026-02-12T17:23:08.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:23:08.450 DEBUG:teuthology.orchestra.run.trial195:> true 2026-02-12T17:23:08.613 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial195.front.sepia.ceph.com' 2026-02-12T17:23:08.613 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-677.el9.x86_64"... 2026-02-12T17:23:08.614 DEBUG:teuthology.orchestra.run.trial195:> uname -r 2026-02-12T17:23:08.669 INFO:teuthology.orchestra.run.trial195.stdout:5.14.0-677.el9.x86_64 2026-02-12T17:23:08.670 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-12T17:23:08.670 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-12T17:23:08.670 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-12T17:23:09.671 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2026-02-12T17:23:09.671 DEBUG:teuthology.orchestra.run.trial195:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-12T17:23:09.701 INFO:teuthology.orchestra.run.trial195.stdout:ttyS1 2026-02-12T17:23:09.728 DEBUG:teuthology.parallel:result is None 2026-02-12T17:23:11.230 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:23:11.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:23:14.333 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:23:29.334 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:23:29.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:23:32.445 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:23:36.870 DEBUG:teuthology.orchestra.remote:timed out 2026-02-12T17:23:45.871 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial010.front.sepia.ceph.com' 2026-02-12T17:23:45.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial010.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:23:46.009 DEBUG:teuthology.orchestra.run.trial010:> true 2026-02-12T17:23:46.173 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial010.front.sepia.ceph.com' 2026-02-12T17:23:46.173 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-677.el9.x86_64"... 2026-02-12T17:23:46.174 DEBUG:teuthology.orchestra.run.trial010:> uname -r 2026-02-12T17:23:46.229 INFO:teuthology.orchestra.run.trial010.stdout:5.14.0-677.el9.x86_64 2026-02-12T17:23:46.230 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-12T17:23:46.230 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-12T17:23:46.230 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-12T17:23:47.231 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-12T17:23:47.231 DEBUG:teuthology.orchestra.run.trial010:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-12T17:23:47.260 INFO:teuthology.orchestra.run.trial010.stdout:ttyS1 2026-02-12T17:23:47.282 DEBUG:teuthology.parallel:result is None 2026-02-12T17:23:50.446 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:23:50.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:23:53.501 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:24:14.501 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:24:14.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:24:17.565 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:24:41.565 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:24:41.566 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:24:44.637 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:25:11.638 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:25:11.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:25:14.717 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:25:44.717 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:25:44.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:25:47.805 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:26:20.806 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:26:20.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:26:23.901 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:26:59.902 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:26:59.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:27:03.005 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:27:42.005 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:27:42.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:27:45.117 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:28:15.118 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:28:15.119 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:28:18.205 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:28:18.205 WARNING:teuthology.contextutil:'reconnect to trial104' reached maximum tries (13) after waiting for 300 seconds 2026-02-12T17:28:18.205 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-677.el9.x86_64"... 2026-02-12T17:28:18.205 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:28:18.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:28:21.278 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:28:30.279 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:28:30.280 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:28:33.373 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:28:45.374 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:28:45.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:28:48.477 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:28:51.478 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com' 2026-02-12T17:28:51.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T17:28:54.557 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.104 2026-02-12T17:28:54.557 WARNING:teuthology.contextutil:'reconnect to trial104' reached maximum tries (4) after waiting for 30 seconds 2026-02-12T17:28:54.557 ERROR:teuthology.task.kernel:Saw exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/task/kernel.py", line 710, in wait_for_reboot assert not need_to_install(ctx, client, need_install[client]), \ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/task/kernel.py", line 182, in need_to_install ctx.cluster.only(role).run( File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/remote.py", line 574, in run raise ConnectionError(f'Failed to reconnect to {self.shortname}') ConnectionError: Failed to reconnect to trial104 2026-02-12T17:28:54.560 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/task/kernel.py", line 1374, in task with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/task/kernel.py", line 1481, in process_role wait_for_reboot(ctx, {role: need_version}, timeout, config) File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/task/kernel.py", line 710, in wait_for_reboot assert not need_to_install(ctx, client, need_install[client]), \ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/task/kernel.py", line 182, in need_to_install ctx.cluster.only(role).run( File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/remote.py", line 574, in run raise ConnectionError(f'Failed to reconnect to {self.shortname}') ConnectionError: Failed to reconnect to trial104 2026-02-12T17:28:54.563 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-12T17:28:54.575 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-12T17:28:54.606 INFO:teuthology.run:Summary data: description: rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-1} backoff/normal ceph clusters/{fixed-4} crc-failures/default d-balancer/read mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/{bluestore/{alloc$/{btree} base mem$/{low} onode-segment$/{1M} write$/{random/{compr$/{no$/{no}} random}}}} rados supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects-localized} failure_reason: Failed to reconnect to trial104 owner: scheduled_yuriw@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-12T17:28:54.607 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-12T17:28:54.743 INFO:teuthology.run:FAIL