2023-05-17T15:07:04.438 INFO:root:teuthology version: 0.0.1.dev98+g69caab8 2023-05-17T15:07:04.439 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --verbose --name yuriw-2023-05-17_14:20:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2023-05-17_14:20:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7276711 --description fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} -- /home/teuthworker/archive/yuriw-2023-05-17_14:20:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7276711/orig.config.yaml 2023-05-17T15:07:04.496 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-05-17T15:07:04.589 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-05-17_14:20:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7276711 branch: wip-yuri2-testing-2023-05-15-0810-pacific_2 description: fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} email: null first_in_suite: false job_id: '7276711' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-05-17_14:20:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri2-testing-2023-05-15-0810-pacific_2 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: false mds op complaint time: 180 mds verify scatter: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - but it is still running - objects unfound and apparently lost - MDS_SLOW_METADATA_IO - MDS_TRIM sha1: 20531b61f434caac5e29cea07411fac6911e8e91 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 20531b61f434caac5e29cea07411fac6911e8e91 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri2-testing-2023-05-15-0810-pacific_2 sha1: 20531b61f434caac5e29cea07411fac6911e8e91 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 6166 sha1: 20531b61f434caac5e29cea07411fac6911e8e91 sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri2-testing-2023-05-15-0810-pacific_2 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_20531b61f434caac5e29cea07411fac6911e8e91/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 20531b61f434caac5e29cea07411fac6911e8e91 targets: smithi032.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0= smithi077.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - thrashosds: null - workunit: clients: all: - suites/fsstress.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 69caab8ec6f44c61c991ae693e8e7b8cd859a896 timestamp: 2023-05-17_14:20:33 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-05-17T15:07:04.589 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_20531b61f434caac5e29cea07411fac6911e8e91/qa; will attempt to use it 2023-05-17T15:07:04.590 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_20531b61f434caac5e29cea07411fac6911e8e91/qa/tasks 2023-05-17T15:07:04.590 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-05-17T15:07:04.592 INFO:teuthology.task.internal:Checking packages... 2023-05-17T15:07:04.623 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '20531b61f434caac5e29cea07411fac6911e8e91' 2023-05-17T15:07:04.623 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-05-17T15:07:04.623 INFO:teuthology.packaging:ref: None 2023-05-17T15:07:04.624 INFO:teuthology.packaging:tag: None 2023-05-17T15:07:04.624 INFO:teuthology.packaging:branch: wip-yuri2-testing-2023-05-15-0810-pacific_2 2023-05-17T15:07:04.624 INFO:teuthology.packaging:sha1: 20531b61f434caac5e29cea07411fac6911e8e91 2023-05-17T15:07:04.624 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri2-testing-2023-05-15-0810-pacific_2 2023-05-17T15:07:04.782 INFO:teuthology.task.internal:Found packages for ceph version 16.2.13-145-g20531b61-1focal 2023-05-17T15:07:04.782 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-05-17T15:07:04.788 INFO:teuthology.task.internal:no buildpackages task found 2023-05-17T15:07:04.788 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-05-17T15:07:04.811 INFO:teuthology.task.internal:Saving configuration 2023-05-17T15:07:04.830 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-05-17T15:07:04.835 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-05-17T15:07:04.864 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-05-17_14:20:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7276711', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-05-17 15:01:09.467097', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-05-17T15:07:04.891 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-05-17_14:20:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7276711', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-05-17 15:01:09.465795', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-05-17T15:07:04.892 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-05-17T15:07:04.905 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2023-05-17T15:07:04.905 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2023-05-17T15:07:04.906 INFO:teuthology.run_tasks:Running task console_log... 2023-05-17T15:07:04.985 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4b8fd20ee0>, signals=[15]) 2023-05-17T15:07:04.986 INFO:teuthology.run_tasks:Running task internal.connect... 2023-05-17T15:07:04.992 INFO:teuthology.task.internal:Opening connections... 2023-05-17T15:07:04.992 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2023-05-17T15:07:04.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-05-17T15:07:05.049 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi077.front.sepia.ceph.com 2023-05-17T15:07:05.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-05-17T15:07:05.103 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-05-17T15:07:05.110 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2023-05-17T15:07:05.133 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2023-05-17T15:07:05.133 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2023-05-17T15:07:05.180 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Ubuntu" 2023-05-17T15:07:05.180 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-05-17T15:07:05.180 INFO:teuthology.orchestra.run.smithi032.stdout:ID=ubuntu 2023-05-17T15:07:05.180 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE=debian 2023-05-17T15:07:05.180 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-05-17T15:07:05.181 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="20.04" 2023-05-17T15:07:05.181 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.ubuntu.com/" 2023-05-17T15:07:05.181 INFO:teuthology.orchestra.run.smithi032.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-05-17T15:07:05.181 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-05-17T15:07:05.181 INFO:teuthology.orchestra.run.smithi032.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-05-17T15:07:05.181 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_CODENAME=focal 2023-05-17T15:07:05.181 INFO:teuthology.orchestra.run.smithi032.stdout:UBUNTU_CODENAME=focal 2023-05-17T15:07:05.182 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2023-05-17T15:07:05.206 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2023-05-17T15:07:05.213 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2023-05-17T15:07:05.213 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2023-05-17T15:07:05.261 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="Ubuntu" 2023-05-17T15:07:05.261 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-05-17T15:07:05.261 INFO:teuthology.orchestra.run.smithi077.stdout:ID=ubuntu 2023-05-17T15:07:05.262 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE=debian 2023-05-17T15:07:05.262 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-05-17T15:07:05.262 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="20.04" 2023-05-17T15:07:05.262 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://www.ubuntu.com/" 2023-05-17T15:07:05.262 INFO:teuthology.orchestra.run.smithi077.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-05-17T15:07:05.262 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-05-17T15:07:05.262 INFO:teuthology.orchestra.run.smithi077.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-05-17T15:07:05.263 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_CODENAME=focal 2023-05-17T15:07:05.263 INFO:teuthology.orchestra.run.smithi077.stdout:UBUNTU_CODENAME=focal 2023-05-17T15:07:05.263 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2023-05-17T15:07:05.290 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-05-17T15:07:05.299 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-05-17T15:07:05.309 INFO:teuthology.task.internal:Checking for old test directory... 2023-05-17T15:07:05.309 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2023-05-17T15:07:05.312 DEBUG:teuthology.orchestra.run.smithi077:> test '!' -e /home/ubuntu/cephtest 2023-05-17T15:07:05.318 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-05-17T15:07:05.326 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-05-17T15:07:05.326 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2023-05-17T15:07:05.328 DEBUG:teuthology.orchestra.run.smithi077:> test -z $(ls -A /var/lib/ceph) 2023-05-17T15:07:05.349 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-05-17T15:07:05.414 INFO:teuthology.run_tasks:Running task kernel... 2023-05-17T15:07:05.431 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-05-17T15:07:05.432 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-05-17T15:07:05.433 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2023-05-17T15:07:05.433 DEBUG:teuthology.orchestra.run.smithi077:> test -f /run/.containerenv -o -f /.dockerenv 2023-05-17T15:07:05.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-17T15:07:05.439 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2023-05-17T15:07:05.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-17T15:07:05.442 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2023-05-17T15:07:05.447 INFO:teuthology.orchestra.run.smithi032.stdout:5.4.0-124-generic 2023-05-17T15:07:05.448 INFO:teuthology.task.kernel:Running kernel on smithi032: 5.4.0-124-generic 2023-05-17T15:07:05.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get clean 2023-05-17T15:07:05.449 INFO:teuthology.orchestra.run.smithi077.stdout:5.4.0-124-generic 2023-05-17T15:07:05.449 INFO:teuthology.task.kernel:Running kernel on smithi077: 5.4.0-124-generic 2023-05-17T15:07:05.449 DEBUG:teuthology.orchestra.run.smithi077:> sudo apt-get clean 2023-05-17T15:07:05.595 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2023-05-17T15:07:05.600 DEBUG:teuthology.orchestra.run.smithi077:> sudo apt-get update 2023-05-17T15:07:05.745 INFO:teuthology.orchestra.run.smithi077.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-05-17T15:07:05.745 INFO:teuthology.orchestra.run.smithi077.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-05-17T15:07:05.745 INFO:teuthology.orchestra.run.smithi077.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-05-17T15:07:05.754 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-05-17T15:07:05.814 INFO:teuthology.orchestra.run.smithi077.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-05-17T15:07:05.886 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-05-17T15:07:05.952 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-05-17T15:07:06.110 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,163 kB] 2023-05-17T15:07:06.125 INFO:teuthology.orchestra.run.smithi077.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [819 kB] 2023-05-17T15:07:06.205 INFO:teuthology.orchestra.run.smithi077.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,543 kB] 2023-05-17T15:07:06.247 INFO:teuthology.orchestra.run.smithi077.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [430 kB] 2023-05-17T15:07:06.249 INFO:teuthology.orchestra.run.smithi077.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [16.5 kB] 2023-05-17T15:07:06.249 INFO:teuthology.orchestra.run.smithi077.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [31.6 kB] 2023-05-17T15:07:06.250 INFO:teuthology.orchestra.run.smithi077.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,846 kB] 2023-05-17T15:07:06.270 INFO:teuthology.orchestra.run.smithi077.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [260 kB] 2023-05-17T15:07:06.273 INFO:teuthology.orchestra.run.smithi077.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [636 B] 2023-05-17T15:07:06.308 INFO:teuthology.orchestra.run.smithi077.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [727 kB] 2023-05-17T15:07:06.362 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-05-17T15:07:06.383 INFO:teuthology.orchestra.run.smithi077.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [591 kB] 2023-05-17T15:07:06.430 INFO:teuthology.orchestra.run.smithi077.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,061 kB] 2023-05-17T15:07:06.452 INFO:teuthology.orchestra.run.smithi077.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [251 kB] 2023-05-17T15:07:06.456 INFO:teuthology.orchestra.run.smithi077.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [24.3 kB] 2023-05-17T15:07:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-05-17T15:07:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.2 kB] 2023-05-17T15:07:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,408 B] 2023-05-17T15:07:06.458 INFO:teuthology.orchestra.run.smithi077.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [612 B] 2023-05-17T15:07:06.458 INFO:teuthology.orchestra.run.smithi077.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-05-17T15:07:06.458 INFO:teuthology.orchestra.run.smithi077.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-05-17T15:07:06.459 INFO:teuthology.orchestra.run.smithi077.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-05-17T15:07:06.494 INFO:teuthology.orchestra.run.smithi077.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-05-17T15:07:06.528 INFO:teuthology.orchestra.run.smithi077.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [24.9 kB] 2023-05-17T15:07:06.538 INFO:teuthology.orchestra.run.smithi077.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-05-17T15:07:06.611 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,543 kB] 2023-05-17T15:07:07.469 INFO:teuthology.orchestra.run.smithi077.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,163 kB] 2023-05-17T15:07:09.830 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [591 kB] 2023-05-17T15:07:10.337 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [819 kB] 2023-05-17T15:07:10.764 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [350 kB] 2023-05-17T15:07:11.139 INFO:teuthology.orchestra.run.smithi077.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [350 kB] 2023-05-17T15:07:11.388 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [12.8 kB] 2023-05-17T15:07:11.412 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [30.3 kB] 2023-05-17T15:07:11.465 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [430 kB] 2023-05-17T15:07:11.472 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,739 kB] 2023-05-17T15:07:11.902 INFO:teuthology.orchestra.run.smithi077.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [12.8 kB] 2023-05-17T15:07:11.933 INFO:teuthology.orchestra.run.smithi077.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,739 kB] 2023-05-17T15:07:12.074 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [16.5 kB] 2023-05-17T15:07:12.101 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,846 kB] 2023-05-17T15:07:14.535 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [244 kB] 2023-05-17T15:07:14.937 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [640 B] 2023-05-17T15:07:14.937 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [596 kB] 2023-05-17T15:07:14.938 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [31.6 kB] 2023-05-17T15:07:14.990 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [260 kB] 2023-05-17T15:07:15.316 INFO:teuthology.orchestra.run.smithi077.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [30.3 kB] 2023-05-17T15:07:15.370 INFO:teuthology.orchestra.run.smithi077.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [244 kB] 2023-05-17T15:07:15.484 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [636 B] 2023-05-17T15:07:15.649 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [727 kB] 2023-05-17T15:07:15.858 INFO:teuthology.orchestra.run.smithi077.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [640 B] 2023-05-17T15:07:15.861 INFO:teuthology.orchestra.run.smithi077.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [596 kB] 2023-05-17T15:07:15.936 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [834 kB] 2023-05-17T15:07:16.867 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,061 kB] 2023-05-17T15:07:17.080 INFO:teuthology.orchestra.run.smithi077.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [834 kB] 2023-05-17T15:07:17.303 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [170 kB] 2023-05-17T15:07:17.944 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [17.7 kB] 2023-05-17T15:07:18.006 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2023-05-17T15:07:18.016 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [22.9 kB] 2023-05-17T15:07:18.087 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,488 B] 2023-05-17T15:07:18.094 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2023-05-17T15:07:18.687 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [251 kB] 2023-05-17T15:07:18.920 INFO:teuthology.orchestra.run.smithi077.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [170 kB] 2023-05-17T15:07:19.042 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [24.3 kB] 2023-05-17T15:07:19.088 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.2 kB] 2023-05-17T15:07:19.114 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-05-17T15:07:19.130 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,408 B] 2023-05-17T15:07:19.134 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [612 B] 2023-05-17T15:07:19.134 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-05-17T15:07:19.190 INFO:teuthology.orchestra.run.smithi077.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [17.7 kB] 2023-05-17T15:07:19.193 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-05-17T15:07:19.209 INFO:teuthology.orchestra.run.smithi077.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2023-05-17T15:07:19.218 INFO:teuthology.orchestra.run.smithi077.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [22.9 kB] 2023-05-17T15:07:19.239 INFO:teuthology.orchestra.run.smithi077.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,488 B] 2023-05-17T15:07:19.244 INFO:teuthology.orchestra.run.smithi077.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2023-05-17T15:07:19.258 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-05-17T15:07:19.424 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-05-17T15:07:19.604 INFO:teuthology.orchestra.run.smithi032.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [24.9 kB] 2023-05-17T15:07:19.648 INFO:teuthology.orchestra.run.smithi032.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-05-17T15:07:21.196 INFO:teuthology.orchestra.run.smithi077.stdout:Fetched 15.3 MB in 14s (1,127 kB/s) 2023-05-17T15:07:21.643 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 15.3 MB in 14s (1,094 kB/s) 2023-05-17T15:07:22.575 INFO:teuthology.orchestra.run.smithi077.stdout:Reading package lists... 2023-05-17T15:07:22.600 DEBUG:teuthology.orchestra.run.smithi077:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-05-17T15:07:22.672 INFO:teuthology.orchestra.run.smithi077.stdout:Reading package lists... 2023-05-17T15:07:22.865 INFO:teuthology.orchestra.run.smithi077.stdout:Building dependency tree... 2023-05-17T15:07:22.866 INFO:teuthology.orchestra.run.smithi077.stdout:Reading state information... 2023-05-17T15:07:23.049 INFO:teuthology.orchestra.run.smithi077.stdout:The following additional packages will be installed: 2023-05-17T15:07:23.049 INFO:teuthology.orchestra.run.smithi077.stdout: linux-generic linux-headers-5.4.0-148 linux-headers-5.4.0-148-generic 2023-05-17T15:07:23.050 INFO:teuthology.orchestra.run.smithi077.stdout: linux-headers-generic linux-image-5.4.0-148-generic 2023-05-17T15:07:23.050 INFO:teuthology.orchestra.run.smithi077.stdout: linux-modules-5.4.0-148-generic linux-modules-extra-5.4.0-148-generic 2023-05-17T15:07:23.050 INFO:teuthology.orchestra.run.smithi077.stdout:Suggested packages: 2023-05-17T15:07:23.050 INFO:teuthology.orchestra.run.smithi077.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-05-17T15:07:23.051 INFO:teuthology.orchestra.run.smithi077.stdout:Recommended packages: 2023-05-17T15:07:23.051 INFO:teuthology.orchestra.run.smithi077.stdout: thermald 2023-05-17T15:07:23.063 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2023-05-17T15:07:23.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-05-17T15:07:23.103 INFO:teuthology.orchestra.run.smithi077.stdout:The following NEW packages will be installed: 2023-05-17T15:07:23.104 INFO:teuthology.orchestra.run.smithi077.stdout: linux-headers-5.4.0-148 linux-headers-5.4.0-148-generic 2023-05-17T15:07:23.104 INFO:teuthology.orchestra.run.smithi077.stdout: linux-image-5.4.0-148-generic linux-modules-5.4.0-148-generic 2023-05-17T15:07:23.104 INFO:teuthology.orchestra.run.smithi077.stdout: linux-modules-extra-5.4.0-148-generic 2023-05-17T15:07:23.106 INFO:teuthology.orchestra.run.smithi077.stdout:The following packages will be upgraded: 2023-05-17T15:07:23.106 INFO:teuthology.orchestra.run.smithi077.stdout: linux-generic linux-headers-generic linux-image-generic 2023-05-17T15:07:23.162 INFO:teuthology.orchestra.run.smithi077.stdout:3 upgraded, 5 newly installed, 0 to remove and 181 not upgraded. 2023-05-17T15:07:23.163 INFO:teuthology.orchestra.run.smithi077.stdout:Need to get 77.1 MB of archives. 2023-05-17T15:07:23.163 INFO:teuthology.orchestra.run.smithi077.stdout:After this operation, 380 MB of additional disk space will be used. 2023-05-17T15:07:23.163 INFO:teuthology.orchestra.run.smithi077.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-148-generic amd64 5.4.0-148.165 [15.0 MB] 2023-05-17T15:07:23.168 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2023-05-17T15:07:23.365 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2023-05-17T15:07:23.367 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2023-05-17T15:07:23.562 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2023-05-17T15:07:23.563 INFO:teuthology.orchestra.run.smithi032.stdout: linux-generic linux-headers-5.4.0-148 linux-headers-5.4.0-148-generic 2023-05-17T15:07:23.563 INFO:teuthology.orchestra.run.smithi032.stdout: linux-headers-generic linux-image-5.4.0-148-generic 2023-05-17T15:07:23.563 INFO:teuthology.orchestra.run.smithi032.stdout: linux-modules-5.4.0-148-generic linux-modules-extra-5.4.0-148-generic 2023-05-17T15:07:23.564 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2023-05-17T15:07:23.564 INFO:teuthology.orchestra.run.smithi032.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-05-17T15:07:23.564 INFO:teuthology.orchestra.run.smithi032.stdout:Recommended packages: 2023-05-17T15:07:23.564 INFO:teuthology.orchestra.run.smithi032.stdout: thermald 2023-05-17T15:07:23.620 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2023-05-17T15:07:23.621 INFO:teuthology.orchestra.run.smithi032.stdout: linux-headers-5.4.0-148 linux-headers-5.4.0-148-generic 2023-05-17T15:07:23.621 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-5.4.0-148-generic linux-modules-5.4.0-148-generic 2023-05-17T15:07:23.621 INFO:teuthology.orchestra.run.smithi032.stdout: linux-modules-extra-5.4.0-148-generic 2023-05-17T15:07:23.622 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages will be upgraded: 2023-05-17T15:07:23.622 INFO:teuthology.orchestra.run.smithi032.stdout: linux-generic linux-headers-generic linux-image-generic 2023-05-17T15:07:23.675 INFO:teuthology.orchestra.run.smithi032.stdout:3 upgraded, 5 newly installed, 0 to remove and 181 not upgraded. 2023-05-17T15:07:23.675 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 77.1 MB of archives. 2023-05-17T15:07:23.675 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 380 MB of additional disk space will be used. 2023-05-17T15:07:23.675 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-148-generic amd64 5.4.0-148.165 [15.0 MB] 2023-05-17T15:07:23.977 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-148-generic amd64 5.4.0-148.165 [10.5 MB] 2023-05-17T15:07:24.069 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-148-generic amd64 5.4.0-148.165 [39.2 MB] 2023-05-17T15:07:24.535 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.148.146 [1,900 B] 2023-05-17T15:07:24.535 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.148.146 [2,496 B] 2023-05-17T15:07:24.536 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-148 all 5.4.0-148.165 [11.0 MB] 2023-05-17T15:07:24.627 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-148-generic amd64 5.4.0-148.165 [1,368 kB] 2023-05-17T15:07:24.638 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.148.146 [2,372 B] 2023-05-17T15:07:25.310 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 77.1 MB in 1s (76.4 MB/s) 2023-05-17T15:07:25.497 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-modules-5.4.0-148-generic. 2023-05-17T15:07:27.187 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-05-17T15:07:27.193 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../0-linux-modules-5.4.0-148-generic_5.4.0-148.165_amd64.deb ... 2023-05-17T15:07:27.433 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-modules-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:07:29.389 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-image-5.4.0-148-generic. 2023-05-17T15:07:29.405 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../1-linux-image-5.4.0-148-generic_5.4.0-148.165_amd64.deb ... 2023-05-17T15:07:29.497 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-image-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:07:30.742 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-148-generic. 2023-05-17T15:07:30.755 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-148-generic_5.4.0-148.165_amd64.deb ... 2023-05-17T15:07:30.800 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-modules-extra-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:07:35.629 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../3-linux-generic_5.4.0.148.146_amd64.deb ... 2023-05-17T15:07:35.780 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-generic (5.4.0.148.146) over (5.4.0.124.125) ... 2023-05-17T15:07:36.147 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.148.146_amd64.deb ... 2023-05-17T15:07:36.300 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-image-generic (5.4.0.148.146) over (5.4.0.124.125) ... 2023-05-17T15:07:36.636 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-headers-5.4.0-148. 2023-05-17T15:07:36.653 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../5-linux-headers-5.4.0-148_5.4.0-148.165_all.deb ... 2023-05-17T15:07:36.694 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-headers-5.4.0-148 (5.4.0-148.165) ... 2023-05-17T15:07:42.384 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-headers-5.4.0-148-generic. 2023-05-17T15:07:42.407 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../6-linux-headers-5.4.0-148-generic_5.4.0-148.165_amd64.deb ... 2023-05-17T15:07:42.450 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-headers-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:07:45.792 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.148.146_amd64.deb ... 2023-05-17T15:07:45.920 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-headers-generic (5.4.0.148.146) over (5.4.0.124.125) ... 2023-05-17T15:07:46.269 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-modules-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:07:47.518 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-headers-5.4.0-148 (5.4.0-148.165) ... 2023-05-17T15:07:47.653 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-image-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:07:48.889 INFO:teuthology.orchestra.run.smithi032.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-148-generic 2023-05-17T15:07:48.890 INFO:teuthology.orchestra.run.smithi032.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-148-generic 2023-05-17T15:07:49.030 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-headers-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:07:49.156 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-modules-extra-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:07:49.680 INFO:teuthology.orchestra.run.smithi077.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-148-generic amd64 5.4.0-148.165 [10.5 MB] 2023-05-17T15:07:50.408 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-image-generic (5.4.0.148.146) ... 2023-05-17T15:07:50.559 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-headers-generic (5.4.0.148.146) ... 2023-05-17T15:07:50.685 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-generic (5.4.0.148.146) ... 2023-05-17T15:07:50.812 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for linux-image-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:07:50.885 INFO:teuthology.orchestra.run.smithi032.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-05-17T15:07:50.885 INFO:teuthology.orchestra.run.smithi032.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-148-generic 2023-05-17T15:08:07.938 INFO:teuthology.orchestra.run.smithi077.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-148-generic amd64 5.4.0-148.165 [39.2 MB] 2023-05-17T15:08:13.508 INFO:teuthology.orchestra.run.smithi032.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-05-17T15:08:13.509 INFO:teuthology.orchestra.run.smithi032.stdout:Sourcing file `/etc/default/grub' 2023-05-17T15:08:13.519 INFO:teuthology.orchestra.run.smithi032.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-05-17T15:08:13.542 INFO:teuthology.orchestra.run.smithi032.stdout:Generating grub configuration file ... 2023-05-17T15:08:13.940 INFO:teuthology.orchestra.run.smithi032.stdout:Found linux image: /boot/vmlinuz-5.4.0-148-generic 2023-05-17T15:08:13.965 INFO:teuthology.orchestra.run.smithi032.stdout:Found initrd image: /boot/initrd.img-5.4.0-148-generic 2023-05-17T15:08:14.364 INFO:teuthology.orchestra.run.smithi032.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-05-17T15:08:14.376 INFO:teuthology.orchestra.run.smithi032.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-05-17T15:08:16.502 INFO:teuthology.orchestra.run.smithi032.stdout:done 2023-05-17T15:08:16.909 DEBUG:teuthology.orchestra.run.smithi032:> dpkg -s linux-image-generic 2023-05-17T15:08:16.969 INFO:teuthology.orchestra.run.smithi032.stdout:Package: linux-image-generic 2023-05-17T15:08:16.969 INFO:teuthology.orchestra.run.smithi032.stdout:Status: install ok installed 2023-05-17T15:08:16.969 INFO:teuthology.orchestra.run.smithi032.stdout:Priority: optional 2023-05-17T15:08:16.969 INFO:teuthology.orchestra.run.smithi032.stdout:Section: kernel 2023-05-17T15:08:16.969 INFO:teuthology.orchestra.run.smithi032.stdout:Installed-Size: 19 2023-05-17T15:08:16.970 INFO:teuthology.orchestra.run.smithi032.stdout:Maintainer: Ubuntu Kernel Team 2023-05-17T15:08:16.970 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: amd64 2023-05-17T15:08:16.970 INFO:teuthology.orchestra.run.smithi032.stdout:Source: linux-meta 2023-05-17T15:08:16.970 INFO:teuthology.orchestra.run.smithi032.stdout:Version: 5.4.0.148.146 2023-05-17T15:08:16.970 INFO:teuthology.orchestra.run.smithi032.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2023-05-17T15:08:16.971 INFO:teuthology.orchestra.run.smithi032.stdout:Depends: linux-image-5.4.0-148-generic, linux-modules-extra-5.4.0-148-generic, linux-firmware, intel-microcode, amd64-microcode 2023-05-17T15:08:16.971 INFO:teuthology.orchestra.run.smithi032.stdout:Recommends: thermald 2023-05-17T15:08:16.971 INFO:teuthology.orchestra.run.smithi032.stdout:Description: Generic Linux kernel image 2023-05-17T15:08:16.971 INFO:teuthology.orchestra.run.smithi032.stdout: This package will always depend on the latest generic kernel image 2023-05-17T15:08:16.971 INFO:teuthology.orchestra.run.smithi032.stdout: available. 2023-05-17T15:08:16.971 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-148-generic 2023-05-17T15:08:16.972 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-05-17T15:08:16.972 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-05-17T15:08:16.972 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-05-17T15:08:16.972 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi032.front.sepia.ceph.com, path=None, version=distro) 2023-05-17T15:08:16.972 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get clean 2023-05-17T15:08:17.069 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2023-05-17T15:08:17.347 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-05-17T15:08:17.347 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-05-17T15:08:17.417 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-05-17T15:08:17.509 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-05-17T15:08:19.073 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2023-05-17T15:08:19.099 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-05-17T15:08:19.166 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2023-05-17T15:08:19.362 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2023-05-17T15:08:19.363 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2023-05-17T15:08:19.577 INFO:teuthology.orchestra.run.smithi032.stdout:linux-image-generic is already the newest version (5.4.0.148.146). 2023-05-17T15:08:19.578 INFO:teuthology.orchestra.run.smithi032.stdout:0 upgraded, 0 newly installed, 0 to remove and 181 not upgraded. 2023-05-17T15:08:19.579 DEBUG:teuthology.orchestra.run.smithi032:> dpkg -s linux-image-generic 2023-05-17T15:08:19.597 INFO:teuthology.orchestra.run.smithi032.stdout:Package: linux-image-generic 2023-05-17T15:08:19.597 INFO:teuthology.orchestra.run.smithi032.stdout:Status: install ok installed 2023-05-17T15:08:19.597 INFO:teuthology.orchestra.run.smithi032.stdout:Priority: optional 2023-05-17T15:08:19.597 INFO:teuthology.orchestra.run.smithi032.stdout:Section: kernel 2023-05-17T15:08:19.598 INFO:teuthology.orchestra.run.smithi032.stdout:Installed-Size: 19 2023-05-17T15:08:19.598 INFO:teuthology.orchestra.run.smithi032.stdout:Maintainer: Ubuntu Kernel Team 2023-05-17T15:08:19.598 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: amd64 2023-05-17T15:08:19.598 INFO:teuthology.orchestra.run.smithi032.stdout:Source: linux-meta 2023-05-17T15:08:19.598 INFO:teuthology.orchestra.run.smithi032.stdout:Version: 5.4.0.148.146 2023-05-17T15:08:19.598 INFO:teuthology.orchestra.run.smithi032.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2023-05-17T15:08:19.598 INFO:teuthology.orchestra.run.smithi032.stdout:Depends: linux-image-5.4.0-148-generic, linux-modules-extra-5.4.0-148-generic, linux-firmware, intel-microcode, amd64-microcode 2023-05-17T15:08:19.599 INFO:teuthology.orchestra.run.smithi032.stdout:Recommends: thermald 2023-05-17T15:08:19.599 INFO:teuthology.orchestra.run.smithi032.stdout:Description: Generic Linux kernel image 2023-05-17T15:08:19.599 INFO:teuthology.orchestra.run.smithi032.stdout: This package will always depend on the latest generic kernel image 2023-05-17T15:08:19.599 INFO:teuthology.orchestra.run.smithi032.stdout: available. 2023-05-17T15:08:19.599 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2023-05-17T15:08:19.606 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.h4QRhMXnqz 2023-05-17T15:08:19.606 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /boot/grub/grub.cfg /tmp/tmp.h4QRhMXnqz 2023-05-17T15:08:19.668 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.h4QRhMXnqz 2023-05-17T15:08:19.702 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.h4QRhMXnqz is 9KB 2023-05-17T15:08:19.714 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.h4QRhMXnqz 2023-05-17T15:08:19.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-05-17T15:08:19.777 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-05-17T15:08:19.778 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-05-17T15:08:19.778 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-05-17T15:08:19.822 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-148-generic 2023-05-17T15:08:19.823 DEBUG:teuthology.orchestra.run.smithi032:> sudo update-grub 2023-05-17T15:08:21.136 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub' 2023-05-17T15:08:21.141 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-05-17T15:08:21.149 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2023-05-17T15:08:21.447 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.4.0-148-generic 2023-05-17T15:08:21.459 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.4.0-148-generic 2023-05-17T15:08:21.806 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-05-17T15:08:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-05-17T15:08:22.828 INFO:teuthology.orchestra.run.smithi032.stderr:done 2023-05-17T15:08:22.830 DEBUG:teuthology.orchestra.run.smithi032:> sudo shutdown -r now 2023-05-17T15:08:52.834 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-05-17T15:08:52.835 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-05-17T15:08:52.837 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-05-17T15:09:09.855 INFO:teuthology.orchestra.run.smithi077.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.148.146 [1,900 B] 2023-05-17T15:09:09.856 INFO:teuthology.orchestra.run.smithi077.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.148.146 [2,496 B] 2023-05-17T15:09:09.856 INFO:teuthology.orchestra.run.smithi077.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-148 all 5.4.0-148.165 [11.0 MB] 2023-05-17T15:09:26.316 INFO:teuthology.orchestra.run.smithi077.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-148-generic amd64 5.4.0-148.165 [1,368 kB] 2023-05-17T15:09:28.370 INFO:teuthology.orchestra.run.smithi077.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.148.146 [2,372 B] 2023-05-17T15:09:29.106 INFO:teuthology.orchestra.run.smithi077.stdout:Fetched 77.1 MB in 2min 5s (616 kB/s) 2023-05-17T15:09:29.301 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package linux-modules-5.4.0-148-generic. 2023-05-17T15:09:30.994 INFO:teuthology.orchestra.run.smithi077.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-05-17T15:09:30.999 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../0-linux-modules-5.4.0-148-generic_5.4.0-148.165_amd64.deb ... 2023-05-17T15:09:31.090 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking linux-modules-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:09:33.066 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package linux-image-5.4.0-148-generic. 2023-05-17T15:09:33.081 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../1-linux-image-5.4.0-148-generic_5.4.0-148.165_amd64.deb ... 2023-05-17T15:09:33.186 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking linux-image-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:09:34.261 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-148-generic. 2023-05-17T15:09:34.276 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-148-generic_5.4.0-148.165_amd64.deb ... 2023-05-17T15:09:34.311 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking linux-modules-extra-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:09:39.125 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../3-linux-generic_5.4.0.148.146_amd64.deb ... 2023-05-17T15:09:39.249 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking linux-generic (5.4.0.148.146) over (5.4.0.124.125) ... 2023-05-17T15:09:39.575 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.148.146_amd64.deb ... 2023-05-17T15:09:39.678 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking linux-image-generic (5.4.0.148.146) over (5.4.0.124.125) ... 2023-05-17T15:09:40.039 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package linux-headers-5.4.0-148. 2023-05-17T15:09:40.054 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../5-linux-headers-5.4.0-148_5.4.0-148.165_all.deb ... 2023-05-17T15:09:40.097 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking linux-headers-5.4.0-148 (5.4.0-148.165) ... 2023-05-17T15:09:45.411 INFO:teuthology.orchestra.run.smithi077.stdout:Selecting previously unselected package linux-headers-5.4.0-148-generic. 2023-05-17T15:09:45.431 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../6-linux-headers-5.4.0-148-generic_5.4.0-148.165_amd64.deb ... 2023-05-17T15:09:45.469 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking linux-headers-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:09:48.161 INFO:teuthology.orchestra.run.smithi077.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.148.146_amd64.deb ... 2023-05-17T15:09:48.262 INFO:teuthology.orchestra.run.smithi077.stdout:Unpacking linux-headers-generic (5.4.0.148.146) over (5.4.0.124.125) ... 2023-05-17T15:09:48.622 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up linux-modules-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:09:49.852 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up linux-headers-5.4.0-148 (5.4.0-148.165) ... 2023-05-17T15:09:49.978 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up linux-image-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:09:51.128 INFO:teuthology.orchestra.run.smithi077.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-148-generic 2023-05-17T15:09:51.129 INFO:teuthology.orchestra.run.smithi077.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-148-generic 2023-05-17T15:09:51.255 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up linux-headers-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:09:51.373 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up linux-modules-extra-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:09:52.591 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up linux-image-generic (5.4.0.148.146) ... 2023-05-17T15:09:52.717 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up linux-headers-generic (5.4.0.148.146) ... 2023-05-17T15:09:52.838 DEBUG:teuthology.orchestra.remote:timed out 2023-05-17T15:09:52.843 INFO:teuthology.orchestra.run.smithi077.stdout:Setting up linux-generic (5.4.0.148.146) ... 2023-05-17T15:09:52.970 INFO:teuthology.orchestra.run.smithi077.stdout:Processing triggers for linux-image-5.4.0-148-generic (5.4.0-148.165) ... 2023-05-17T15:09:53.049 INFO:teuthology.orchestra.run.smithi077.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-05-17T15:09:53.050 INFO:teuthology.orchestra.run.smithi077.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-148-generic 2023-05-17T15:10:14.831 INFO:teuthology.orchestra.run.smithi077.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-05-17T15:10:14.831 INFO:teuthology.orchestra.run.smithi077.stdout:Sourcing file `/etc/default/grub' 2023-05-17T15:10:14.852 INFO:teuthology.orchestra.run.smithi077.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-05-17T15:10:14.872 INFO:teuthology.orchestra.run.smithi077.stdout:Generating grub configuration file ... 2023-05-17T15:10:15.186 INFO:teuthology.orchestra.run.smithi077.stdout:Found linux image: /boot/vmlinuz-5.4.0-148-generic 2023-05-17T15:10:15.213 INFO:teuthology.orchestra.run.smithi077.stdout:Found initrd image: /boot/initrd.img-5.4.0-148-generic 2023-05-17T15:10:15.630 INFO:teuthology.orchestra.run.smithi077.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-05-17T15:10:15.642 INFO:teuthology.orchestra.run.smithi077.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-05-17T15:10:17.684 INFO:teuthology.orchestra.run.smithi077.stdout:done 2023-05-17T15:10:18.141 DEBUG:teuthology.orchestra.run.smithi077:> dpkg -s linux-image-generic 2023-05-17T15:10:18.188 INFO:teuthology.orchestra.run.smithi077.stdout:Package: linux-image-generic 2023-05-17T15:10:18.189 INFO:teuthology.orchestra.run.smithi077.stdout:Status: install ok installed 2023-05-17T15:10:18.189 INFO:teuthology.orchestra.run.smithi077.stdout:Priority: optional 2023-05-17T15:10:18.189 INFO:teuthology.orchestra.run.smithi077.stdout:Section: kernel 2023-05-17T15:10:18.189 INFO:teuthology.orchestra.run.smithi077.stdout:Installed-Size: 19 2023-05-17T15:10:18.189 INFO:teuthology.orchestra.run.smithi077.stdout:Maintainer: Ubuntu Kernel Team 2023-05-17T15:10:18.190 INFO:teuthology.orchestra.run.smithi077.stdout:Architecture: amd64 2023-05-17T15:10:18.190 INFO:teuthology.orchestra.run.smithi077.stdout:Source: linux-meta 2023-05-17T15:10:18.190 INFO:teuthology.orchestra.run.smithi077.stdout:Version: 5.4.0.148.146 2023-05-17T15:10:18.190 INFO:teuthology.orchestra.run.smithi077.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2023-05-17T15:10:18.190 INFO:teuthology.orchestra.run.smithi077.stdout:Depends: linux-image-5.4.0-148-generic, linux-modules-extra-5.4.0-148-generic, linux-firmware, intel-microcode, amd64-microcode 2023-05-17T15:10:18.190 INFO:teuthology.orchestra.run.smithi077.stdout:Recommends: thermald 2023-05-17T15:10:18.190 INFO:teuthology.orchestra.run.smithi077.stdout:Description: Generic Linux kernel image 2023-05-17T15:10:18.190 INFO:teuthology.orchestra.run.smithi077.stdout: This package will always depend on the latest generic kernel image 2023-05-17T15:10:18.191 INFO:teuthology.orchestra.run.smithi077.stdout: available. 2023-05-17T15:10:18.191 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-148-generic 2023-05-17T15:10:18.191 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-05-17T15:10:18.191 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-05-17T15:10:18.191 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-05-17T15:10:18.192 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi077.front.sepia.ceph.com, path=None, version=distro) 2023-05-17T15:10:18.192 DEBUG:teuthology.orchestra.run.smithi077:> sudo apt-get clean 2023-05-17T15:10:18.273 DEBUG:teuthology.orchestra.run.smithi077:> sudo apt-get update 2023-05-17T15:10:18.534 INFO:teuthology.orchestra.run.smithi077.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-05-17T15:10:18.535 INFO:teuthology.orchestra.run.smithi077.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-05-17T15:10:18.604 INFO:teuthology.orchestra.run.smithi077.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-05-17T15:10:18.687 INFO:teuthology.orchestra.run.smithi077.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-05-17T15:10:20.220 INFO:teuthology.orchestra.run.smithi077.stdout:Reading package lists... 2023-05-17T15:10:20.245 DEBUG:teuthology.orchestra.run.smithi077:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-05-17T15:10:20.311 INFO:teuthology.orchestra.run.smithi077.stdout:Reading package lists... 2023-05-17T15:10:20.502 INFO:teuthology.orchestra.run.smithi077.stdout:Building dependency tree... 2023-05-17T15:10:20.503 INFO:teuthology.orchestra.run.smithi077.stdout:Reading state information... 2023-05-17T15:10:20.702 INFO:teuthology.orchestra.run.smithi077.stdout:linux-image-generic is already the newest version (5.4.0.148.146). 2023-05-17T15:10:20.703 INFO:teuthology.orchestra.run.smithi077.stdout:0 upgraded, 0 newly installed, 0 to remove and 181 not upgraded. 2023-05-17T15:10:20.704 DEBUG:teuthology.orchestra.run.smithi077:> dpkg -s linux-image-generic 2023-05-17T15:10:20.727 INFO:teuthology.orchestra.run.smithi077.stdout:Package: linux-image-generic 2023-05-17T15:10:20.727 INFO:teuthology.orchestra.run.smithi077.stdout:Status: install ok installed 2023-05-17T15:10:20.727 INFO:teuthology.orchestra.run.smithi077.stdout:Priority: optional 2023-05-17T15:10:20.727 INFO:teuthology.orchestra.run.smithi077.stdout:Section: kernel 2023-05-17T15:10:20.727 INFO:teuthology.orchestra.run.smithi077.stdout:Installed-Size: 19 2023-05-17T15:10:20.728 INFO:teuthology.orchestra.run.smithi077.stdout:Maintainer: Ubuntu Kernel Team 2023-05-17T15:10:20.728 INFO:teuthology.orchestra.run.smithi077.stdout:Architecture: amd64 2023-05-17T15:10:20.728 INFO:teuthology.orchestra.run.smithi077.stdout:Source: linux-meta 2023-05-17T15:10:20.728 INFO:teuthology.orchestra.run.smithi077.stdout:Version: 5.4.0.148.146 2023-05-17T15:10:20.728 INFO:teuthology.orchestra.run.smithi077.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2023-05-17T15:10:20.728 INFO:teuthology.orchestra.run.smithi077.stdout:Depends: linux-image-5.4.0-148-generic, linux-modules-extra-5.4.0-148-generic, linux-firmware, intel-microcode, amd64-microcode 2023-05-17T15:10:20.728 INFO:teuthology.orchestra.run.smithi077.stdout:Recommends: thermald 2023-05-17T15:10:20.729 INFO:teuthology.orchestra.run.smithi077.stdout:Description: Generic Linux kernel image 2023-05-17T15:10:20.729 INFO:teuthology.orchestra.run.smithi077.stdout: This package will always depend on the latest generic kernel image 2023-05-17T15:10:20.729 INFO:teuthology.orchestra.run.smithi077.stdout: available. 2023-05-17T15:10:20.729 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2023-05-17T15:10:20.733 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.5aMywuGJns 2023-05-17T15:10:20.733 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /boot/grub/grub.cfg /tmp/tmp.5aMywuGJns 2023-05-17T15:10:20.791 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.5aMywuGJns 2023-05-17T15:10:20.829 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.5aMywuGJns is 9KB 2023-05-17T15:10:20.842 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.5aMywuGJns 2023-05-17T15:10:20.848 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-05-17T15:10:20.911 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-05-17T15:10:20.911 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-05-17T15:10:20.911 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-05-17T15:10:20.946 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-148-generic 2023-05-17T15:10:20.946 DEBUG:teuthology.orchestra.run.smithi077:> sudo update-grub 2023-05-17T15:10:22.190 INFO:teuthology.orchestra.run.smithi077.stderr:Sourcing file `/etc/default/grub' 2023-05-17T15:10:22.198 INFO:teuthology.orchestra.run.smithi077.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-05-17T15:10:22.213 INFO:teuthology.orchestra.run.smithi077.stderr:Generating grub configuration file ... 2023-05-17T15:10:22.485 INFO:teuthology.orchestra.run.smithi077.stderr:Found linux image: /boot/vmlinuz-5.4.0-148-generic 2023-05-17T15:10:22.497 INFO:teuthology.orchestra.run.smithi077.stderr:Found initrd image: /boot/initrd.img-5.4.0-148-generic 2023-05-17T15:10:22.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-05-17T15:10:22.841 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-05-17T15:10:22.849 INFO:teuthology.orchestra.run.smithi077.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-05-17T15:10:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-05-17T15:10:23.049 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-05-17T15:10:23.640 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-05-17T15:10:23.641 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-148-generic"... 2023-05-17T15:10:23.641 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2023-05-17T15:10:23.647 INFO:teuthology.orchestra.run.smithi032.stdout:5.4.0-148-generic 2023-05-17T15:10:23.647 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-148-generic vs 5.4.0-148-generic 2023-05-17T15:10:23.647 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-05-17T15:10:23.648 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-05-17T15:10:23.907 INFO:teuthology.orchestra.run.smithi077.stderr:done 2023-05-17T15:10:23.908 DEBUG:teuthology.orchestra.run.smithi077:> sudo shutdown -r now 2023-05-17T15:10:24.649 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-05-17T15:10:24.650 DEBUG:teuthology.orchestra.run.smithi032:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-05-17T15:10:24.758 INFO:teuthology.orchestra.run.smithi032.stdout:ttyS1 2023-05-17T15:10:24.781 DEBUG:teuthology.parallel:result is None 2023-05-17T15:10:53.911 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-05-17T15:10:53.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2023-05-17T15:10:53.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-05-17T15:11:53.915 DEBUG:teuthology.orchestra.remote:timed out 2023-05-17T15:12:23.917 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2023-05-17T15:12:23.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-05-17T15:12:24.126 DEBUG:teuthology.orchestra.run.smithi077:> true 2023-05-17T15:12:24.690 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2023-05-17T15:12:24.691 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-148-generic"... 2023-05-17T15:12:24.691 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2023-05-17T15:12:24.697 INFO:teuthology.orchestra.run.smithi077.stdout:5.4.0-148-generic 2023-05-17T15:12:24.697 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-148-generic vs 5.4.0-148-generic 2023-05-17T15:12:24.698 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-05-17T15:12:24.698 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-05-17T15:12:25.698 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-05-17T15:12:25.699 DEBUG:teuthology.orchestra.run.smithi077:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-05-17T15:12:25.786 INFO:teuthology.orchestra.run.smithi077.stdout:ttyS1 2023-05-17T15:12:25.818 DEBUG:teuthology.parallel:result is None 2023-05-17T15:12:25.819 INFO:teuthology.run_tasks:Running task internal.base... 2023-05-17T15:12:25.828 INFO:teuthology.task.internal:Creating test directory... 2023-05-17T15:12:25.829 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-05-17T15:12:25.832 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-05-17T15:12:25.838 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-05-17T15:12:25.847 INFO:teuthology.run_tasks:Running task internal.archive... 2023-05-17T15:12:25.858 INFO:teuthology.task.internal:Creating archive directory... 2023-05-17T15:12:25.859 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-05-17T15:12:25.861 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-05-17T15:12:25.886 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-05-17T15:12:25.894 INFO:teuthology.task.internal:Enabling coredump saving... 2023-05-17T15:12:25.895 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-05-17T15:12:25.916 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-05-17T15:12:25.936 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-05-17T15:12:25.938 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-05-17T15:12:25.946 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-05-17T15:12:25.948 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-05-17T15:12:25.950 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-05-17T15:12:25.958 INFO:teuthology.task.internal:Configuring sudo... 2023-05-17T15:12:25.958 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-05-17T15:12:25.960 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-05-17T15:12:25.975 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-05-17T15:12:25.987 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-05-17T15:12:25.988 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-05-17T15:12:25.992 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-05-17T15:12:25.998 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-17T15:12:26.045 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-17T15:12:26.050 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-05-17T15:12:26.051 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-05-17T15:12:26.072 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-05-17T15:12:26.079 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-05-17T15:12:26.086 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-05-17T15:12:26.086 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-05-17T15:12:26.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2023-05-17T15:12:26.116 DEBUG:teuthology.orchestra.run.smithi077:> sudo service rsyslog restart 2023-05-17T15:12:26.195 INFO:teuthology.run_tasks:Running task internal.timer... 2023-05-17T15:12:26.202 INFO:teuthology.task.internal:Starting timer... 2023-05-17T15:12:26.203 INFO:teuthology.run_tasks:Running task pcp... 2023-05-17T15:12:26.215 INFO:teuthology.run_tasks:Running task selinux... 2023-05-17T15:12:26.226 DEBUG:teuthology.task.selinux:Excluding smithi032: OS 'ubuntu' does not support SELinux 2023-05-17T15:12:26.227 DEBUG:teuthology.task.selinux:Excluding smithi077: OS 'ubuntu' does not support SELinux 2023-05-17T15:12:26.227 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-05-17T15:12:26.227 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-05-17T15:12:26.228 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-05-17T15:12:26.228 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-05-17T15:12:26.239 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-05-17T15:12:26.239 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-05-17T15:12:26.262 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-05-17T15:12:26.264 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi077.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-05-17T15:19:25.000 INFO:teuthology.task.ansible:Archiving ansible failure log at: /home/teuthworker/archive/yuriw-2023-05-17_14:20:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7276711/ansible_failures.yaml 2023-05-17T15:19:25.024 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/task/__init__.py", line 123, in __enter__ self.begin() File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/task/ansible.py", line 426, in begin super(CephLab, self).begin() File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/task/ansible.py", line 267, in begin self.execute_playbook() File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/task/ansible.py", line 295, in execute_playbook self._handle_failure(command, status) File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/task/ansible.py", line 320, in _handle_failure raise AnsibleFailedError(failures) teuthology.exceptions.AnsibleFailedError: {'smithi077.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_unauthenticated': False, 'autoclean': False, 'autoremove': False, 'cache_valid_time': 0, 'deb': None, 'default_release': None, 'dpkg_options': 'force-confdef,force-confold', 'force': True, 'force_apt_get': False, 'install_recommends': None, 'name': ['ceph', 'ceph-common', 'libcephfs1', 'radosgw', 'python-ceph', 'python-rados', 'python-cephfs', 'python-rbd', 'librbd1', 'librados2', 'ceph-fs-common-dbg', 'ceph-fs-common', 'openmpi-common'], 'only_upgrade': False, 'package': ['ceph', 'ceph-common', 'libcephfs1', 'radosgw', 'python-ceph', 'python-rados', 'python-cephfs', 'python-rbd', 'librbd1', 'librados2', 'ceph-fs-common-dbg', 'ceph-fs-common', 'openmpi-common'], 'policy_rc_d': None, 'purge': False, 'state': 'absent', 'update_cache': None, 'update_cache_retries': 5, 'update_cache_retry_max_delay': 12, 'upgrade': None}}, 'msg': "'apt-get remove 'librbd1' 'librados2'' failed: W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 14791 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n", 'rc': 100, 'stderr': 'W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 14791 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'stderr_lines': ['W: --force-yes is deprecated, use one of the options starting with --allow instead.', 'E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 14791 (apt-get)', 'E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?'], 'stdout': '', 'stdout_lines': []}} 2023-05-17T15:19:25.158 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=05ce0fd0fba243a29661bc936e18d7a3 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/task/__init__.py", line 123, in __enter__ self.begin() File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/task/ansible.py", line 426, in begin super(CephLab, self).begin() File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/task/ansible.py", line 267, in begin self.execute_playbook() File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/task/ansible.py", line 295, in execute_playbook self._handle_failure(command, status) File "/home/teuthworker/src/git.ceph.com_teuthology_69caab8ec6f44c61c991ae693e8e7b8cd859a896/teuthology/task/ansible.py", line 320, in _handle_failure raise AnsibleFailedError(failures) teuthology.exceptions.AnsibleFailedError: {'smithi077.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_unauthenticated': False, 'autoclean': False, 'autoremove': False, 'cache_valid_time': 0, 'deb': None, 'default_release': None, 'dpkg_options': 'force-confdef,force-confold', 'force': True, 'force_apt_get': False, 'install_recommends': None, 'name': ['ceph', 'ceph-common', 'libcephfs1', 'radosgw', 'python-ceph', 'python-rados', 'python-cephfs', 'python-rbd', 'librbd1', 'librados2', 'ceph-fs-common-dbg', 'ceph-fs-common', 'openmpi-common'], 'only_upgrade': False, 'package': ['ceph', 'ceph-common', 'libcephfs1', 'radosgw', 'python-ceph', 'python-rados', 'python-cephfs', 'python-rbd', 'librbd1', 'librados2', 'ceph-fs-common-dbg', 'ceph-fs-common', 'openmpi-common'], 'policy_rc_d': None, 'purge': False, 'state': 'absent', 'update_cache': None, 'update_cache_retries': 5, 'update_cache_retry_max_delay': 12, 'upgrade': None}}, 'msg': "'apt-get remove 'librbd1' 'librados2'' failed: W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 14791 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n", 'rc': 100, 'stderr': 'W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 14791 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'stderr_lines': ['W: --force-yes is deprecated, use one of the options starting with --allow instead.', 'E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 14791 (apt-get)', 'E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?'], 'stdout': '', 'stdout_lines': []}} 2023-05-17T15:19:25.161 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-05-17T15:19:25.169 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-05-17T15:19:25.169 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-05-17T15:19:25.189 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-05-17T15:19:25.196 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-05-17T15:19:25.204 INFO:teuthology.task.internal:Duration was 419.001455 seconds 2023-05-17T15:19:25.205 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-05-17T15:19:25.212 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-05-17T15:19:25.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-05-17T15:19:25.233 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-05-17T15:19:25.286 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-05-17T15:19:25.286 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2023-05-17T15:19:25.287 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-05-17T15:19:25.301 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi077.front.sepia.ceph.com 2023-05-17T15:19:25.301 DEBUG:teuthology.orchestra.run.smithi077:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-05-17T15:19:25.316 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-05-17T15:19:25.317 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-05-17T15:19:25.319 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-05-17T15:19:25.401 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-05-17T15:19:25.402 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-05-17T15:19:25.404 DEBUG:teuthology.orchestra.run.smithi077:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-05-17T15:19:25.521 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-05-17T15:19:25.530 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-05-17T15:19:25.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-05-17T15:19:25.545 DEBUG:teuthology.orchestra.run.smithi077:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-05-17T15:19:25.561 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-05-17T15:19:25.570 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-05-17T15:19:25.572 DEBUG:teuthology.orchestra.run.smithi077:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-05-17T15:19:25.585 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2023-05-17T15:19:25.619 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = core 2023-05-17T15:19:25.641 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2023-05-17T15:19:25.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-17T15:19:25.663 DEBUG:teuthology.orchestra.run.smithi077:> test -e /home/ubuntu/cephtest/archive/coredump 2023-05-17T15:19:25.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-05-17T15:19:25.668 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-05-17T15:19:25.677 INFO:teuthology.task.internal:Transferring archived files... 2023-05-17T15:19:25.679 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-05-17_14:20:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7276711/remote/smithi032 2023-05-17T15:19:25.680 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-05-17T15:19:25.702 DEBUG:teuthology.misc:Transferring archived files from smithi077:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-05-17_14:20:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7276711/remote/smithi077 2023-05-17T15:19:25.703 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-05-17T15:19:25.733 INFO:teuthology.task.internal:Removing archive directory... 2023-05-17T15:19:25.734 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2023-05-17T15:19:25.752 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf -- /home/ubuntu/cephtest/archive 2023-05-17T15:19:25.758 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-05-17T15:19:25.768 INFO:teuthology.task.internal:Not uploading archives. 2023-05-17T15:19:25.768 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-05-17T15:19:25.777 INFO:teuthology.task.internal:Tidying up after the test... 2023-05-17T15:19:25.777 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-05-17T15:19:25.780 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-05-17T15:19:25.785 INFO:teuthology.orchestra.run.smithi032.stdout: 924115 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 17 15:19 /home/ubuntu/cephtest 2023-05-17T15:19:25.787 INFO:teuthology.orchestra.run.smithi077.stdout: 924113 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 17 15:19 /home/ubuntu/cephtest 2023-05-17T15:19:25.789 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-05-17T15:19:25.802 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-05-17T15:19:25.821 INFO:teuthology.nuke:Checking targets against current locks 2023-05-17T15:19:25.852 DEBUG:teuthology.nuke:shortname: smithi032 2023-05-17T15:19:25.853 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-05-17T15:19:25.882 DEBUG:teuthology.nuke:shortname: smithi077 2023-05-17T15:19:25.883 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-05-17T15:19:25.886 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-05-17_14:20:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7276711', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-05-17 15:01:09.467097', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0='} 2023-05-17T15:19:25.911 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-05-17_14:20:33-fs-wip-yuri2-testing-2023-05-15-0810-pacific_2-distro-default-smithi/7276711', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-05-17 15:01:09.465795', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0='} 2023-05-17T15:19:25.944 INFO:teuthology.orchestra.console:Power off smithi032 2023-05-17T15:19:25.945 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-05-17T15:19:25.970 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-05-17T15:19:25.992 INFO:teuthology.orchestra.console:Power off smithi077 2023-05-17T15:19:25.993 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-05-17T15:19:26.016 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-05-17T15:19:30.006 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-05-17T15:19:30.040 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-05-17T15:19:34.122 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-05-17T15:19:34.153 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-05-17T15:19:34.334 INFO:teuthology.orchestra.console:Power off for smithi032 completed 2023-05-17T15:19:38.268 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-05-17T15:19:38.481 INFO:teuthology.orchestra.console:Power off for smithi077 completed 2023-05-17T15:19:38.999 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} duration: 419.0014545917511 failure_reason: '{''smithi077.front.sepia.ceph.com'': {''_ansible_no_log'': False, ''changed'': False, ''invocation'': {''module_args'': {''allow_unauthenticated'': False, ''autoclean'': False, ''autoremove'': False, ''cache_valid_time'': 0, ''deb'': None, ''default_release'': None, ''dpkg_options'': ''force-confdef,force-confold'', ''force'': True, ''force_apt_get'': False, ''install_recommends'': None, ''name'': [''ceph'', ''ceph-common'', ''libcephfs1'', ''radosgw'', ''python-ceph'', ''python-rados'', ''python-cephfs'', ''python-rbd'', ''librbd1'', ''librados2'', ''ceph-fs-common-dbg'', ''ceph-fs-common'', ''openmpi-common''], ''only_upgrade'': False, ''package'': [''ceph'', ''ceph-common'', ''libcephfs1'', ''radosgw'', ''python-ceph'', ''python-rados'', ''python-cephfs'', ''python-rbd'', ''librbd1'', ''librados2'', ''ceph-fs-common-dbg'', ''ceph-fs-common'', ''openmpi-common''], ''policy_rc_d'': None, ''purge'': False, ''state'': ''absent'', ''update_cache'': None, ''update_cache_retries'': 5, ''update_cache_retry_max_delay'': 12, ''upgrade'': None}}, ''msg'': "''apt-get remove ''librbd1'' ''librados2'''' failed: W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 14791 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n", ''rc'': 100, ''stderr'': ''W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 14791 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n'', ''stderr_lines'': [''W: --force-yes is deprecated, use one of the options starting with --allow instead.'', ''E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 14791 (apt-get)'', ''E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?''], ''stdout'': '''', ''stdout_lines'': []}}' owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=05ce0fd0fba243a29661bc936e18d7a3 status: dead success: false 2023-05-17T15:19:39.000 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-05-17T15:19:39.117 INFO:teuthology.run:DEAD