2022-11-01T02:11:10.885 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9 2022-11-01T02:11:10.886 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} --archive /home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080309 --owner scheduled_teuthology@teuthology --verbose -- /home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080309/orig.config.yaml 2022-11-01T02:11:10.936 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-11-01T02:11:11.005 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080309 branch: quincy description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} email: ceph-qa@ceph.io first_in_suite: false job_id: '7080309' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: quincy ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 fuse set user groups: true rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: false mds op complaint time: 180 mds verify scatter: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - but it is still running - objects unfound and apparently lost - MDS_SLOW_METADATA_IO sha1: 5219abe5bdb882abcc3a550aa02e563f2cd638bd ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 5219abe5bdb882abcc3a550aa02e563f2cd638bd thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: quincy sha1: 5fbb8171010367eef3729cd64d88a414f76e7d7a owner: scheduled_teuthology@teuthology priority: 1000 repo: git://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 sha1: 5219abe5bdb882abcc3a550aa02e563f2cd638bd sleep_before_teardown: 0 suite: fs suite_branch: quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph_5fbb8171010367eef3729cd64d88a414f76e7d7a/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph.git suite_sha1: 5fbb8171010367eef3729cd64d88a414f76e7d7a targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA+yEveplGC9BAXjnUshJ7Vitr+3a9UQ8pCvcpwhFoMXMcD4udh8roV4Zbd+35UpXqCH3+reCA1bOeFm/J/hUJo= smithi133.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkhq0aEnRIKmOTWPAy3QwbDBZqZfRf8/1TDKde4naEjnBGLQaP8ZOVzFOYFmZ2uP2YLSib8PvNNLlMTacey5S4oVrCML5jTuA1iv8kW+umRvZfs+03taeMMJM2FR+adK+eCT1ZiJYwDgxqQDR6GXU/JQMfGKCpv4zPaCaiPzndiZDl70Z1IwtA9ajWtwCooo4/MLcdobkpqCt5pNdnx/mVSaSIG01bxAT3JhvTmn3xZAM50u8ITbnkQpiSEWlcWK4MQuQJihGvjApUb3vx1ovj+s1LR1WRWl1NJz9tWSTG30LGE367sujAMsEXfXfj3PxzvToavJu7vSB+ocJWKOQLKEr8Yg+V6ivOVBMoe33h4iN6Xsk9f9RRM+kw63zOoWUGMQBMqdbe8unqi3iBCzil95l3pAk5lHmsxVzGiiubg//Htx/yY6U7Q6/gNs4ZgeBc3vXnO4vEKnrii5QYPlC/RnBYiaW55yOwvV7qmWvun4qwYfhhUwjo6DiOUsViNEc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - thrashosds: null - workunit: clients: all: - suites/pjd.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 35ea38a9840006713a3d42472a2c536a25e88c15 timestamp: 2022-10-25_03:15:06 tube: smithi user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.18616 2022-11-01T02:11:11.006 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_5fbb8171010367eef3729cd64d88a414f76e7d7a/qa; will attempt to use it 2022-11-01T02:11:11.007 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_5fbb8171010367eef3729cd64d88a414f76e7d7a/qa/tasks 2022-11-01T02:11:11.007 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-11-01T02:11:11.009 INFO:teuthology.task.internal:Checking packages... 2022-11-01T02:11:11.033 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '5219abe5bdb882abcc3a550aa02e563f2cd638bd' 2022-11-01T02:11:11.033 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-01T02:11:11.034 INFO:teuthology.packaging:ref: None 2022-11-01T02:11:11.034 INFO:teuthology.packaging:tag: None 2022-11-01T02:11:11.034 INFO:teuthology.packaging:branch: quincy 2022-11-01T02:11:11.035 INFO:teuthology.packaging:sha1: 5219abe5bdb882abcc3a550aa02e563f2cd638bd 2022-11-01T02:11:11.035 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=quincy 2022-11-01T02:11:11.474 INFO:teuthology.task.internal:Found packages for ceph version 17.2.5-264-ge15b63d2-1focal 2022-11-01T02:11:11.474 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-11-01T02:11:11.480 INFO:teuthology.task.internal:no buildpackages task found 2022-11-01T02:11:11.481 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-11-01T02:11:11.520 INFO:teuthology.task.internal:Saving configuration 2022-11-01T02:11:11.539 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-11-01T02:11:11.544 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-11-01T02:11:11.568 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080309', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-01 02:05:12.318613', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICxn6wYHoLoy/tBrvX6Uj4MKrupmlOofAsksBGJ2vN/O'} 2022-11-01T02:11:11.590 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2022-10-25_03:15:06-fs-quincy-distro-default-smithi/7080309', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-01 02:05:12.317249', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEezuG3Pn1LycGY2QsNIF/+Kma4rZWvXhXdc7QTCysaV'} 2022-11-01T02:11:11.591 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-11-01T02:11:11.597 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-11-01T02:11:11.597 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-11-01T02:11:11.597 INFO:teuthology.run_tasks:Running task console_log... 2022-11-01T02:11:11.682 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faf883677b8>, signals=[15]) 2022-11-01T02:11:11.683 INFO:teuthology.run_tasks:Running task internal.connect... 2022-11-01T02:11:11.689 INFO:teuthology.task.internal:Opening connections... 2022-11-01T02:11:11.690 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2022-11-01T02:11:11.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-01T02:11:11.786 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2022-11-01T02:11:11.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-01T02:11:11.877 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-11-01T02:11:11.884 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2022-11-01T02:11:11.891 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2022-11-01T02:11:11.891 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2022-11-01T02:11:11.940 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Ubuntu" 2022-11-01T02:11:11.941 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="20.04.5 LTS (Focal Fossa)" 2022-11-01T02:11:11.941 INFO:teuthology.orchestra.run.smithi012.stdout:ID=ubuntu 2022-11-01T02:11:11.941 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE=debian 2022-11-01T02:11:11.941 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Ubuntu 20.04.5 LTS" 2022-11-01T02:11:11.942 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="20.04" 2022-11-01T02:11:11.942 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.ubuntu.com/" 2022-11-01T02:11:11.942 INFO:teuthology.orchestra.run.smithi012.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-11-01T02:11:11.943 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-11-01T02:11:11.943 INFO:teuthology.orchestra.run.smithi012.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-11-01T02:11:11.943 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_CODENAME=focal 2022-11-01T02:11:11.943 INFO:teuthology.orchestra.run.smithi012.stdout:UBUNTU_CODENAME=focal 2022-11-01T02:11:11.944 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2022-11-01T02:11:11.964 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2022-11-01T02:11:11.971 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2022-11-01T02:11:11.971 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2022-11-01T02:11:11.978 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="Ubuntu" 2022-11-01T02:11:11.978 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="20.04.5 LTS (Focal Fossa)" 2022-11-01T02:11:11.978 INFO:teuthology.orchestra.run.smithi133.stdout:ID=ubuntu 2022-11-01T02:11:11.978 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE=debian 2022-11-01T02:11:11.979 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="Ubuntu 20.04.5 LTS" 2022-11-01T02:11:11.979 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="20.04" 2022-11-01T02:11:11.979 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://www.ubuntu.com/" 2022-11-01T02:11:11.979 INFO:teuthology.orchestra.run.smithi133.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-11-01T02:11:11.979 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-11-01T02:11:11.980 INFO:teuthology.orchestra.run.smithi133.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-11-01T02:11:11.980 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_CODENAME=focal 2022-11-01T02:11:11.980 INFO:teuthology.orchestra.run.smithi133.stdout:UBUNTU_CODENAME=focal 2022-11-01T02:11:11.981 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2022-11-01T02:11:11.997 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-11-01T02:11:12.007 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-11-01T02:11:12.054 INFO:teuthology.task.internal:Checking for old test directory... 2022-11-01T02:11:12.055 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2022-11-01T02:11:12.057 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2022-11-01T02:11:12.063 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-11-01T02:11:12.070 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-11-01T02:11:12.070 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2022-11-01T02:11:12.072 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2022-11-01T02:11:12.115 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-11-01T02:11:12.169 INFO:teuthology.run_tasks:Running task kernel... 2022-11-01T02:11:12.184 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2022-11-01T02:11:12.185 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2022-11-01T02:11:12.186 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-01T02:11:12.187 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-01T02:11:12.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-01T02:11:12.192 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2022-11-01T02:11:12.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-01T02:11:12.194 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2022-11-01T02:11:12.201 INFO:teuthology.orchestra.run.smithi133.stdout:5.4.0-128-generic 2022-11-01T02:11:12.201 INFO:teuthology.orchestra.run.smithi012.stdout:5.4.0-128-generic 2022-11-01T02:11:12.202 INFO:teuthology.task.kernel:Running kernel on smithi133: 5.4.0-128-generic 2022-11-01T02:11:12.202 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get clean 2022-11-01T02:11:12.203 INFO:teuthology.task.kernel:Running kernel on smithi012: 5.4.0-128-generic 2022-11-01T02:11:12.203 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get clean 2022-11-01T02:11:12.313 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2022-11-01T02:11:12.372 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get update 2022-11-01T02:11:12.464 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-11-01T02:11:12.522 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-11-01T02:11:12.533 INFO:teuthology.orchestra.run.smithi133.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-11-01T02:11:12.576 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2022-11-01T02:11:12.595 INFO:teuthology.orchestra.run.smithi133.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-11-01T02:11:12.633 INFO:teuthology.orchestra.run.smithi133.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-11-01T02:11:12.656 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-11-01T02:11:12.765 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [514 kB] 2022-11-01T02:11:12.837 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,821 kB] 2022-11-01T02:11:12.874 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [301 kB] 2022-11-01T02:11:12.877 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [11.2 kB] 2022-11-01T02:11:12.878 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,289 kB] 2022-11-01T02:11:12.891 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [183 kB] 2022-11-01T02:11:12.892 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [566 kB] 2022-11-01T02:11:12.896 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [743 kB] 2022-11-01T02:11:12.900 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [137 kB] 2022-11-01T02:11:12.901 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [15.3 kB] 2022-11-01T02:11:12.903 INFO:teuthology.orchestra.run.smithi133.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,196 kB] 2022-11-01T02:11:13.002 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-11-01T02:11:13.012 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [745 kB] 2022-11-01T02:11:13.026 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [385 kB] 2022-11-01T02:11:13.028 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [16.0 kB] 2022-11-01T02:11:13.028 INFO:teuthology.orchestra.run.smithi133.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [26.6 kB] 2022-11-01T02:11:13.029 INFO:teuthology.orchestra.run.smithi133.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,381 kB] 2022-11-01T02:11:13.039 INFO:teuthology.orchestra.run.smithi133.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [196 kB] 2022-11-01T02:11:13.041 INFO:teuthology.orchestra.run.smithi133.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [600 B] 2022-11-01T02:11:13.041 INFO:teuthology.orchestra.run.smithi133.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [697 kB] 2022-11-01T02:11:13.047 INFO:teuthology.orchestra.run.smithi133.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [972 kB] 2022-11-01T02:11:13.054 INFO:teuthology.orchestra.run.smithi133.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [221 kB] 2022-11-01T02:11:13.055 INFO:teuthology.orchestra.run.smithi133.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [21.8 kB] 2022-11-01T02:11:13.061 INFO:teuthology.orchestra.run.smithi133.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [9,820 B] 2022-11-01T02:11:13.062 INFO:teuthology.orchestra.run.smithi133.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [29.9 kB] 2022-11-01T02:11:13.062 INFO:teuthology.orchestra.run.smithi133.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,940 B] 2022-11-01T02:11:13.062 INFO:teuthology.orchestra.run.smithi133.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [664 B] 2022-11-01T02:11:13.063 INFO:teuthology.orchestra.run.smithi133.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2022-11-01T02:11:13.063 INFO:teuthology.orchestra.run.smithi133.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2022-11-01T02:11:13.078 INFO:teuthology.orchestra.run.smithi133.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2022-11-01T02:11:13.078 INFO:teuthology.orchestra.run.smithi133.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.5 kB] 2022-11-01T02:11:13.079 INFO:teuthology.orchestra.run.smithi133.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [24.0 kB] 2022-11-01T02:11:13.079 INFO:teuthology.orchestra.run.smithi133.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [864 B] 2022-11-01T02:11:13.143 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [745 kB] 2022-11-01T02:11:13.230 INFO:teuthology.orchestra.run.smithi133.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [514 kB] 2022-11-01T02:11:13.364 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,196 kB] 2022-11-01T02:11:13.520 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [385 kB] 2022-11-01T02:11:13.521 INFO:teuthology.orchestra.run.smithi133.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,821 kB] 2022-11-01T02:11:13.533 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [16.0 kB] 2022-11-01T02:11:13.534 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [26.6 kB] 2022-11-01T02:11:13.534 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,381 kB] 2022-11-01T02:11:13.573 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [196 kB] 2022-11-01T02:11:13.577 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [600 B] 2022-11-01T02:11:13.578 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [972 kB] 2022-11-01T02:11:13.599 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [697 kB] 2022-11-01T02:11:13.621 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [221 kB] 2022-11-01T02:11:13.625 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [21.8 kB] 2022-11-01T02:11:13.658 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [9,820 B] 2022-11-01T02:11:13.659 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [29.9 kB] 2022-11-01T02:11:13.660 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,940 B] 2022-11-01T02:11:13.660 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [664 B] 2022-11-01T02:11:13.660 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2022-11-01T02:11:13.689 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2022-11-01T02:11:13.690 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2022-11-01T02:11:13.696 INFO:teuthology.orchestra.run.smithi133.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [301 kB] 2022-11-01T02:11:13.710 INFO:teuthology.orchestra.run.smithi133.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [11.2 kB] 2022-11-01T02:11:13.711 INFO:teuthology.orchestra.run.smithi133.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,289 kB] 2022-11-01T02:11:13.767 INFO:teuthology.orchestra.run.smithi133.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [183 kB] 2022-11-01T02:11:13.769 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.5 kB] 2022-11-01T02:11:13.770 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [24.0 kB] 2022-11-01T02:11:13.771 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [864 B] 2022-11-01T02:11:13.772 INFO:teuthology.orchestra.run.smithi133.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [566 kB] 2022-11-01T02:11:13.787 INFO:teuthology.orchestra.run.smithi133.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [743 kB] 2022-11-01T02:11:13.804 INFO:teuthology.orchestra.run.smithi133.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [137 kB] 2022-11-01T02:11:13.806 INFO:teuthology.orchestra.run.smithi133.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [15.3 kB] 2022-11-01T02:11:17.591 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 12.9 MB in 2s (6,525 kB/s) 2022-11-01T02:11:17.701 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 12.9 MB in 2s (6,662 kB/s) 2022-11-01T02:11:18.876 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2022-11-01T02:11:18.899 DEBUG:teuthology.orchestra.run.smithi133:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-11-01T02:11:18.968 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2022-11-01T02:11:19.009 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2022-11-01T02:11:19.034 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-11-01T02:11:19.102 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2022-11-01T02:11:19.132 INFO:teuthology.orchestra.run.smithi133.stdout:Building dependency tree... 2022-11-01T02:11:19.132 INFO:teuthology.orchestra.run.smithi133.stdout:Reading state information... 2022-11-01T02:11:19.272 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2022-11-01T02:11:19.273 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2022-11-01T02:11:19.280 INFO:teuthology.orchestra.run.smithi133.stdout:The following additional packages will be installed: 2022-11-01T02:11:19.281 INFO:teuthology.orchestra.run.smithi133.stdout: linux-generic linux-headers-5.4.0-131 linux-headers-5.4.0-131-generic 2022-11-01T02:11:19.281 INFO:teuthology.orchestra.run.smithi133.stdout: linux-headers-generic linux-image-5.4.0-131-generic 2022-11-01T02:11:19.281 INFO:teuthology.orchestra.run.smithi133.stdout: linux-modules-5.4.0-131-generic linux-modules-extra-5.4.0-131-generic 2022-11-01T02:11:19.281 INFO:teuthology.orchestra.run.smithi133.stdout:Suggested packages: 2022-11-01T02:11:19.282 INFO:teuthology.orchestra.run.smithi133.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2022-11-01T02:11:19.282 INFO:teuthology.orchestra.run.smithi133.stdout:Recommended packages: 2022-11-01T02:11:19.282 INFO:teuthology.orchestra.run.smithi133.stdout: thermald 2022-11-01T02:11:19.342 INFO:teuthology.orchestra.run.smithi133.stdout:The following NEW packages will be installed: 2022-11-01T02:11:19.342 INFO:teuthology.orchestra.run.smithi133.stdout: linux-headers-5.4.0-131 linux-headers-5.4.0-131-generic 2022-11-01T02:11:19.342 INFO:teuthology.orchestra.run.smithi133.stdout: linux-image-5.4.0-131-generic linux-modules-5.4.0-131-generic 2022-11-01T02:11:19.343 INFO:teuthology.orchestra.run.smithi133.stdout: linux-modules-extra-5.4.0-131-generic 2022-11-01T02:11:19.344 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages will be upgraded: 2022-11-01T02:11:19.344 INFO:teuthology.orchestra.run.smithi133.stdout: linux-generic linux-headers-generic linux-image-generic 2022-11-01T02:11:19.433 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2022-11-01T02:11:19.433 INFO:teuthology.orchestra.run.smithi012.stdout: linux-generic linux-headers-5.4.0-131 linux-headers-5.4.0-131-generic 2022-11-01T02:11:19.434 INFO:teuthology.orchestra.run.smithi012.stdout: linux-headers-generic linux-image-5.4.0-131-generic 2022-11-01T02:11:19.434 INFO:teuthology.orchestra.run.smithi012.stdout: linux-modules-5.4.0-131-generic linux-modules-extra-5.4.0-131-generic 2022-11-01T02:11:19.434 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2022-11-01T02:11:19.435 INFO:teuthology.orchestra.run.smithi012.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2022-11-01T02:11:19.435 INFO:teuthology.orchestra.run.smithi012.stdout:Recommended packages: 2022-11-01T02:11:19.435 INFO:teuthology.orchestra.run.smithi012.stdout: thermald 2022-11-01T02:11:19.487 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2022-11-01T02:11:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: linux-headers-5.4.0-131 linux-headers-5.4.0-131-generic 2022-11-01T02:11:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.4.0-131-generic linux-modules-5.4.0-131-generic 2022-11-01T02:11:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: linux-modules-extra-5.4.0-131-generic 2022-11-01T02:11:19.489 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages will be upgraded: 2022-11-01T02:11:19.490 INFO:teuthology.orchestra.run.smithi012.stdout: linux-generic linux-headers-generic linux-image-generic 2022-11-01T02:11:19.523 INFO:teuthology.orchestra.run.smithi133.stdout:3 upgraded, 5 newly installed, 0 to remove and 20 not upgraded. 2022-11-01T02:11:19.524 INFO:teuthology.orchestra.run.smithi133.stdout:Need to get 77.0 MB of archives. 2022-11-01T02:11:19.524 INFO:teuthology.orchestra.run.smithi133.stdout:After this operation, 380 MB of additional disk space will be used. 2022-11-01T02:11:19.524 INFO:teuthology.orchestra.run.smithi133.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-131-generic amd64 5.4.0-131.147 [15.0 MB] 2022-11-01T02:11:19.542 INFO:teuthology.orchestra.run.smithi012.stdout:3 upgraded, 5 newly installed, 0 to remove and 20 not upgraded. 2022-11-01T02:11:19.543 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 77.0 MB of archives. 2022-11-01T02:11:19.543 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 380 MB of additional disk space will be used. 2022-11-01T02:11:19.543 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-131-generic amd64 5.4.0-131.147 [15.0 MB] 2022-11-01T02:11:19.864 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-131-generic amd64 5.4.0-131.147 [10.5 MB] 2022-11-01T02:11:19.956 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-131-generic amd64 5.4.0-131.147 [39.2 MB] 2022-11-01T02:11:20.447 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.131.131 [1,900 B] 2022-11-01T02:11:20.448 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.131.131 [2,524 B] 2022-11-01T02:11:20.448 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-131 all 5.4.0-131.147 [11.0 MB] 2022-11-01T02:11:20.543 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-131-generic amd64 5.4.0-131.147 [1,389 kB] 2022-11-01T02:11:20.554 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.131.131 [2,388 B] 2022-11-01T02:11:20.652 INFO:teuthology.orchestra.run.smithi133.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-131-generic amd64 5.4.0-131.147 [10.5 MB] 2022-11-01T02:11:20.919 INFO:teuthology.orchestra.run.smithi133.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-131-generic amd64 5.4.0-131.147 [39.2 MB] 2022-11-01T02:11:20.999 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 77.0 MB in 1s (72.8 MB/s) 2022-11-01T02:11:21.177 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-modules-5.4.0-131-generic. 2022-11-01T02:11:22.157 INFO:teuthology.orchestra.run.smithi133.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.131.131 [1,900 B] 2022-11-01T02:11:22.157 INFO:teuthology.orchestra.run.smithi133.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.131.131 [2,524 B] 2022-11-01T02:11:22.158 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-131 all 5.4.0-131.147 [11.0 MB] 2022-11-01T02:11:22.453 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-131-generic amd64 5.4.0-131.147 [1,389 kB] 2022-11-01T02:11:22.490 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.131.131 [2,388 B] 2022-11-01T02:11:22.500 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102417 files and directories currently installed.) 2022-11-01T02:11:22.505 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../0-linux-modules-5.4.0-131-generic_5.4.0-131.147_amd64.deb ... 2022-11-01T02:11:22.698 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-modules-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:22.912 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 77.0 MB in 3s (24.5 MB/s) 2022-11-01T02:11:23.084 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-modules-5.4.0-131-generic. 2022-11-01T02:11:24.386 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102417 files and directories currently installed.) 2022-11-01T02:11:24.391 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../0-linux-modules-5.4.0-131-generic_5.4.0-131.147_amd64.deb ... 2022-11-01T02:11:24.480 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-modules-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:24.649 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-image-5.4.0-131-generic. 2022-11-01T02:11:24.665 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../1-linux-image-5.4.0-131-generic_5.4.0-131.147_amd64.deb ... 2022-11-01T02:11:24.753 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:25.844 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-131-generic. 2022-11-01T02:11:25.858 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-131-generic_5.4.0-131.147_amd64.deb ... 2022-11-01T02:11:25.902 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-modules-extra-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:26.222 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-image-5.4.0-131-generic. 2022-11-01T02:11:26.235 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../1-linux-image-5.4.0-131-generic_5.4.0-131.147_amd64.deb ... 2022-11-01T02:11:26.330 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-image-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:27.458 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-131-generic. 2022-11-01T02:11:27.472 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-131-generic_5.4.0-131.147_amd64.deb ... 2022-11-01T02:11:27.516 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-modules-extra-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:31.015 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../3-linux-generic_5.4.0.131.131_amd64.deb ... 2022-11-01T02:11:31.141 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-generic (5.4.0.131.131) over (5.4.0.128.129) ... 2022-11-01T02:11:31.441 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.131.131_amd64.deb ... 2022-11-01T02:11:31.570 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-generic (5.4.0.131.131) over (5.4.0.128.129) ... 2022-11-01T02:11:31.864 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-headers-5.4.0-131. 2022-11-01T02:11:31.880 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../5-linux-headers-5.4.0-131_5.4.0-131.147_all.deb ... 2022-11-01T02:11:31.922 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-5.4.0-131 (5.4.0-131.147) ... 2022-11-01T02:11:32.267 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../3-linux-generic_5.4.0.131.131_amd64.deb ... 2022-11-01T02:11:32.369 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-generic (5.4.0.131.131) over (5.4.0.128.129) ... 2022-11-01T02:11:32.702 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.131.131_amd64.deb ... 2022-11-01T02:11:32.831 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-image-generic (5.4.0.131.131) over (5.4.0.128.129) ... 2022-11-01T02:11:33.192 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-headers-5.4.0-131. 2022-11-01T02:11:33.206 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../5-linux-headers-5.4.0-131_5.4.0-131.147_all.deb ... 2022-11-01T02:11:33.250 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-headers-5.4.0-131 (5.4.0-131.147) ... 2022-11-01T02:11:37.217 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-headers-5.4.0-131-generic. 2022-11-01T02:11:37.238 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../6-linux-headers-5.4.0-131-generic_5.4.0-131.147_amd64.deb ... 2022-11-01T02:11:37.275 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:38.579 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-headers-5.4.0-131-generic. 2022-11-01T02:11:38.600 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../6-linux-headers-5.4.0-131-generic_5.4.0-131.147_amd64.deb ... 2022-11-01T02:11:38.637 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-headers-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:40.318 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.131.131_amd64.deb ... 2022-11-01T02:11:40.439 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-generic (5.4.0.131.131) over (5.4.0.128.129) ... 2022-11-01T02:11:40.757 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-5.4.0-131 (5.4.0-131.147) ... 2022-11-01T02:11:40.877 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-modules-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:41.145 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.131.131_amd64.deb ... 2022-11-01T02:11:41.317 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-headers-generic (5.4.0.131.131) over (5.4.0.128.129) ... 2022-11-01T02:11:41.826 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-headers-5.4.0-131 (5.4.0-131.147) ... 2022-11-01T02:11:41.938 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-modules-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:42.087 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:42.205 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:43.182 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-headers-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:43.308 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-image-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:43.408 INFO:teuthology.orchestra.run.smithi012.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-131-generic 2022-11-01T02:11:43.408 INFO:teuthology.orchestra.run.smithi012.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-131-generic 2022-11-01T02:11:43.540 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-generic (5.4.0.131.131) ... 2022-11-01T02:11:43.666 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-modules-extra-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:44.440 INFO:teuthology.orchestra.run.smithi133.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-131-generic 2022-11-01T02:11:44.440 INFO:teuthology.orchestra.run.smithi133.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-131-generic 2022-11-01T02:11:44.568 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-headers-generic (5.4.0.131.131) ... 2022-11-01T02:11:44.669 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-modules-extra-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:45.368 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-generic (5.4.0.131.131) ... 2022-11-01T02:11:45.702 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-generic (5.4.0.131.131) ... 2022-11-01T02:11:45.820 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for linux-image-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:45.862 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-image-generic (5.4.0.131.131) ... 2022-11-01T02:11:45.870 INFO:teuthology.orchestra.run.smithi012.stdout:/etc/kernel/postinst.d/initramfs-tools: 2022-11-01T02:11:45.870 INFO:teuthology.orchestra.run.smithi012.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-131-generic 2022-11-01T02:11:45.988 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-generic (5.4.0.131.131) ... 2022-11-01T02:11:46.140 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for linux-image-5.4.0-131-generic (5.4.0-131.147) ... 2022-11-01T02:11:46.207 INFO:teuthology.orchestra.run.smithi133.stdout:/etc/kernel/postinst.d/initramfs-tools: 2022-11-01T02:11:46.208 INFO:teuthology.orchestra.run.smithi133.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-131-generic 2022-11-01T02:12:08.038 INFO:teuthology.orchestra.run.smithi012.stdout:/etc/kernel/postinst.d/zz-update-grub: 2022-11-01T02:12:08.039 INFO:teuthology.orchestra.run.smithi012.stdout:Sourcing file `/etc/default/grub' 2022-11-01T02:12:08.064 INFO:teuthology.orchestra.run.smithi012.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-11-01T02:12:08.079 INFO:teuthology.orchestra.run.smithi012.stdout:Generating grub configuration file ... 2022-11-01T02:12:08.215 INFO:teuthology.orchestra.run.smithi133.stdout:/etc/kernel/postinst.d/zz-update-grub: 2022-11-01T02:12:08.215 INFO:teuthology.orchestra.run.smithi133.stdout:Sourcing file `/etc/default/grub' 2022-11-01T02:12:08.230 INFO:teuthology.orchestra.run.smithi133.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-11-01T02:12:08.242 INFO:teuthology.orchestra.run.smithi133.stdout:Generating grub configuration file ... 2022-11-01T02:12:08.356 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.4.0-131-generic 2022-11-01T02:12:08.385 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.4.0-131-generic 2022-11-01T02:12:08.599 INFO:teuthology.orchestra.run.smithi133.stdout:Found linux image: /boot/vmlinuz-5.4.0-131-generic 2022-11-01T02:12:08.631 INFO:teuthology.orchestra.run.smithi133.stdout:Found initrd image: /boot/initrd.img-5.4.0-131-generic 2022-11-01T02:12:08.739 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.4.0-128-generic 2022-11-01T02:12:08.750 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.4.0-128-generic 2022-11-01T02:12:09.018 INFO:teuthology.orchestra.run.smithi133.stdout:Found linux image: /boot/vmlinuz-5.4.0-128-generic 2022-11-01T02:12:09.030 INFO:teuthology.orchestra.run.smithi133.stdout:Found initrd image: /boot/initrd.img-5.4.0-128-generic 2022-11-01T02:12:11.362 INFO:teuthology.orchestra.run.smithi012.stdout:done 2022-11-01T02:12:11.702 INFO:teuthology.orchestra.run.smithi133.stdout:done 2022-11-01T02:12:11.761 DEBUG:teuthology.orchestra.run.smithi012:> dpkg -s linux-image-generic 2022-11-01T02:12:11.792 INFO:teuthology.orchestra.run.smithi012.stdout:Package: linux-image-generic 2022-11-01T02:12:11.793 INFO:teuthology.orchestra.run.smithi012.stdout:Status: install ok installed 2022-11-01T02:12:11.793 INFO:teuthology.orchestra.run.smithi012.stdout:Priority: optional 2022-11-01T02:12:11.793 INFO:teuthology.orchestra.run.smithi012.stdout:Section: kernel 2022-11-01T02:12:11.793 INFO:teuthology.orchestra.run.smithi012.stdout:Installed-Size: 19 2022-11-01T02:12:11.794 INFO:teuthology.orchestra.run.smithi012.stdout:Maintainer: Ubuntu Kernel Team 2022-11-01T02:12:11.794 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: amd64 2022-11-01T02:12:11.794 INFO:teuthology.orchestra.run.smithi012.stdout:Source: linux-meta 2022-11-01T02:12:11.794 INFO:teuthology.orchestra.run.smithi012.stdout:Version: 5.4.0.131.131 2022-11-01T02:12:11.795 INFO:teuthology.orchestra.run.smithi012.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-11-01T02:12:11.795 INFO:teuthology.orchestra.run.smithi012.stdout:Depends: linux-image-5.4.0-131-generic, linux-modules-extra-5.4.0-131-generic, linux-firmware, intel-microcode, amd64-microcode 2022-11-01T02:12:11.795 INFO:teuthology.orchestra.run.smithi012.stdout:Recommends: thermald 2022-11-01T02:12:11.795 INFO:teuthology.orchestra.run.smithi012.stdout:Description: Generic Linux kernel image 2022-11-01T02:12:11.795 INFO:teuthology.orchestra.run.smithi012.stdout: This package will always depend on the latest generic kernel image 2022-11-01T02:12:11.796 INFO:teuthology.orchestra.run.smithi012.stdout: available. 2022-11-01T02:12:11.796 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-128-generic Expected: 5.4.0-131-generic 2022-11-01T02:12:11.796 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2022-11-01T02:12:11.796 DEBUG:teuthology.task.kernel:src is distro, skipping download 2022-11-01T02:12:11.797 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2022-11-01T02:12:11.797 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi012.front.sepia.ceph.com, path=None, version=distro) 2022-11-01T02:12:11.797 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get clean 2022-11-01T02:12:11.872 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2022-11-01T02:12:12.023 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-11-01T02:12:12.084 DEBUG:teuthology.orchestra.run.smithi133:> dpkg -s linux-image-generic 2022-11-01T02:12:12.128 INFO:teuthology.orchestra.run.smithi133.stdout:Package: linux-image-generic 2022-11-01T02:12:12.128 INFO:teuthology.orchestra.run.smithi133.stdout:Status: install ok installed 2022-11-01T02:12:12.128 INFO:teuthology.orchestra.run.smithi133.stdout:Priority: optional 2022-11-01T02:12:12.128 INFO:teuthology.orchestra.run.smithi133.stdout:Section: kernel 2022-11-01T02:12:12.129 INFO:teuthology.orchestra.run.smithi133.stdout:Installed-Size: 19 2022-11-01T02:12:12.129 INFO:teuthology.orchestra.run.smithi133.stdout:Maintainer: Ubuntu Kernel Team 2022-11-01T02:12:12.129 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: amd64 2022-11-01T02:12:12.129 INFO:teuthology.orchestra.run.smithi133.stdout:Source: linux-meta 2022-11-01T02:12:12.129 INFO:teuthology.orchestra.run.smithi133.stdout:Version: 5.4.0.131.131 2022-11-01T02:12:12.130 INFO:teuthology.orchestra.run.smithi133.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-11-01T02:12:12.130 INFO:teuthology.orchestra.run.smithi133.stdout:Depends: linux-image-5.4.0-131-generic, linux-modules-extra-5.4.0-131-generic, linux-firmware, intel-microcode, amd64-microcode 2022-11-01T02:12:12.130 INFO:teuthology.orchestra.run.smithi133.stdout:Recommends: thermald 2022-11-01T02:12:12.130 INFO:teuthology.orchestra.run.smithi133.stdout:Description: Generic Linux kernel image 2022-11-01T02:12:12.131 INFO:teuthology.orchestra.run.smithi133.stdout: This package will always depend on the latest generic kernel image 2022-11-01T02:12:12.131 INFO:teuthology.orchestra.run.smithi133.stdout: available. 2022-11-01T02:12:12.132 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-128-generic Expected: 5.4.0-131-generic 2022-11-01T02:12:12.132 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2022-11-01T02:12:12.132 DEBUG:teuthology.task.kernel:src is distro, skipping download 2022-11-01T02:12:12.132 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2022-11-01T02:12:12.133 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi133.front.sepia.ceph.com, path=None, version=distro) 2022-11-01T02:12:12.133 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get clean 2022-11-01T02:12:12.137 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2022-11-01T02:12:12.196 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get update 2022-11-01T02:12:12.217 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-11-01T02:12:12.297 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-11-01T02:12:12.448 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-11-01T02:12:12.449 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-11-01T02:12:12.519 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-11-01T02:12:12.603 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-11-01T02:12:13.672 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2022-11-01T02:12:13.697 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-11-01T02:12:13.766 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2022-11-01T02:12:13.935 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2022-11-01T02:12:13.936 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2022-11-01T02:12:14.063 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2022-11-01T02:12:14.084 DEBUG:teuthology.orchestra.run.smithi133:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-11-01T02:12:14.130 INFO:teuthology.orchestra.run.smithi012.stdout:linux-image-generic is already the newest version (5.4.0.131.131). 2022-11-01T02:12:14.130 INFO:teuthology.orchestra.run.smithi012.stdout:0 upgraded, 0 newly installed, 0 to remove and 20 not upgraded. 2022-11-01T02:12:14.132 DEBUG:teuthology.orchestra.run.smithi012:> dpkg -s linux-image-generic 2022-11-01T02:12:14.149 INFO:teuthology.orchestra.run.smithi012.stdout:Package: linux-image-generic 2022-11-01T02:12:14.149 INFO:teuthology.orchestra.run.smithi012.stdout:Status: install ok installed 2022-11-01T02:12:14.150 INFO:teuthology.orchestra.run.smithi012.stdout:Priority: optional 2022-11-01T02:12:14.150 INFO:teuthology.orchestra.run.smithi012.stdout:Section: kernel 2022-11-01T02:12:14.150 INFO:teuthology.orchestra.run.smithi012.stdout:Installed-Size: 19 2022-11-01T02:12:14.150 INFO:teuthology.orchestra.run.smithi012.stdout:Maintainer: Ubuntu Kernel Team 2022-11-01T02:12:14.151 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: amd64 2022-11-01T02:12:14.151 INFO:teuthology.orchestra.run.smithi012.stdout:Source: linux-meta 2022-11-01T02:12:14.151 INFO:teuthology.orchestra.run.smithi012.stdout:Version: 5.4.0.131.131 2022-11-01T02:12:14.151 INFO:teuthology.orchestra.run.smithi012.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-11-01T02:12:14.151 INFO:teuthology.orchestra.run.smithi012.stdout:Depends: linux-image-5.4.0-131-generic, linux-modules-extra-5.4.0-131-generic, linux-firmware, intel-microcode, amd64-microcode 2022-11-01T02:12:14.152 INFO:teuthology.orchestra.run.smithi012.stdout:Recommends: thermald 2022-11-01T02:12:14.152 INFO:teuthology.orchestra.run.smithi012.stdout:Description: Generic Linux kernel image 2022-11-01T02:12:14.152 INFO:teuthology.orchestra.run.smithi012.stdout: This package will always depend on the latest generic kernel image 2022-11-01T02:12:14.152 INFO:teuthology.orchestra.run.smithi012.stdout: available. 2022-11-01T02:12:14.153 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2022-11-01T02:12:14.154 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2022-11-01T02:12:14.197 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.yhPCD6biO7 2022-11-01T02:12:14.197 DEBUG:teuthology.orchestra.run.smithi012:> sudo cp /boot/grub/grub.cfg /tmp/tmp.yhPCD6biO7 2022-11-01T02:12:14.213 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.yhPCD6biO7 2022-11-01T02:12:14.250 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.yhPCD6biO7 is 9KB 2022-11-01T02:12:14.265 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.yhPCD6biO7 2022-11-01T02:12:14.271 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2022-11-01T02:12:14.316 INFO:teuthology.orchestra.run.smithi133.stdout:Building dependency tree... 2022-11-01T02:12:14.317 INFO:teuthology.orchestra.run.smithi133.stdout:Reading state information... 2022-11-01T02:12:14.328 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2022-11-01T02:12:14.328 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/grub.d/01_ceph_kernel 2022-11-01T02:12:14.328 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2022-11-01T02:12:14.361 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-131-generic 2022-11-01T02:12:14.361 DEBUG:teuthology.orchestra.run.smithi012:> sudo update-grub 2022-11-01T02:12:14.498 INFO:teuthology.orchestra.run.smithi133.stdout:linux-image-generic is already the newest version (5.4.0.131.131). 2022-11-01T02:12:14.498 INFO:teuthology.orchestra.run.smithi133.stdout:0 upgraded, 0 newly installed, 0 to remove and 20 not upgraded. 2022-11-01T02:12:14.500 DEBUG:teuthology.orchestra.run.smithi133:> dpkg -s linux-image-generic 2022-11-01T02:12:14.522 INFO:teuthology.orchestra.run.smithi133.stdout:Package: linux-image-generic 2022-11-01T02:12:14.523 INFO:teuthology.orchestra.run.smithi133.stdout:Status: install ok installed 2022-11-01T02:12:14.523 INFO:teuthology.orchestra.run.smithi133.stdout:Priority: optional 2022-11-01T02:12:14.523 INFO:teuthology.orchestra.run.smithi133.stdout:Section: kernel 2022-11-01T02:12:14.523 INFO:teuthology.orchestra.run.smithi133.stdout:Installed-Size: 19 2022-11-01T02:12:14.524 INFO:teuthology.orchestra.run.smithi133.stdout:Maintainer: Ubuntu Kernel Team 2022-11-01T02:12:14.524 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: amd64 2022-11-01T02:12:14.524 INFO:teuthology.orchestra.run.smithi133.stdout:Source: linux-meta 2022-11-01T02:12:14.524 INFO:teuthology.orchestra.run.smithi133.stdout:Version: 5.4.0.131.131 2022-11-01T02:12:14.524 INFO:teuthology.orchestra.run.smithi133.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-11-01T02:12:14.525 INFO:teuthology.orchestra.run.smithi133.stdout:Depends: linux-image-5.4.0-131-generic, linux-modules-extra-5.4.0-131-generic, linux-firmware, intel-microcode, amd64-microcode 2022-11-01T02:12:14.525 INFO:teuthology.orchestra.run.smithi133.stdout:Recommends: thermald 2022-11-01T02:12:14.525 INFO:teuthology.orchestra.run.smithi133.stdout:Description: Generic Linux kernel image 2022-11-01T02:12:14.525 INFO:teuthology.orchestra.run.smithi133.stdout: This package will always depend on the latest generic kernel image 2022-11-01T02:12:14.526 INFO:teuthology.orchestra.run.smithi133.stdout: available. 2022-11-01T02:12:14.526 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2022-11-01T02:12:14.531 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.vzFNq8ySCN 2022-11-01T02:12:14.532 DEBUG:teuthology.orchestra.run.smithi133:> sudo cp /boot/grub/grub.cfg /tmp/tmp.vzFNq8ySCN 2022-11-01T02:12:14.599 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.vzFNq8ySCN 2022-11-01T02:12:14.659 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.vzFNq8ySCN is 9KB 2022-11-01T02:12:14.674 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.vzFNq8ySCN 2022-11-01T02:12:14.681 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2022-11-01T02:12:14.738 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2022-11-01T02:12:14.739 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/grub.d/01_ceph_kernel 2022-11-01T02:12:14.739 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2022-11-01T02:12:14.791 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-131-generic 2022-11-01T02:12:14.791 DEBUG:teuthology.orchestra.run.smithi133:> sudo update-grub 2022-11-01T02:12:15.591 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub' 2022-11-01T02:12:15.593 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-11-01T02:12:15.597 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2022-11-01T02:12:15.861 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.4.0-131-generic 2022-11-01T02:12:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.4.0-131-generic 2022-11-01T02:12:16.100 INFO:teuthology.orchestra.run.smithi133.stderr:Sourcing file `/etc/default/grub' 2022-11-01T02:12:16.112 INFO:teuthology.orchestra.run.smithi133.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-11-01T02:12:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2022-11-01T02:12:16.224 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.4.0-128-generic 2022-11-01T02:12:16.236 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.4.0-128-generic 2022-11-01T02:12:16.397 INFO:teuthology.orchestra.run.smithi133.stderr:Found linux image: /boot/vmlinuz-5.4.0-131-generic 2022-11-01T02:12:16.409 INFO:teuthology.orchestra.run.smithi133.stderr:Found initrd image: /boot/initrd.img-5.4.0-131-generic 2022-11-01T02:12:16.811 INFO:teuthology.orchestra.run.smithi133.stderr:Found linux image: /boot/vmlinuz-5.4.0-128-generic 2022-11-01T02:12:16.822 INFO:teuthology.orchestra.run.smithi133.stderr:Found initrd image: /boot/initrd.img-5.4.0-128-generic 2022-11-01T02:12:17.849 INFO:teuthology.orchestra.run.smithi012.stderr:done 2022-11-01T02:12:17.851 DEBUG:teuthology.orchestra.run.smithi012:> sudo shutdown -r now 2022-11-01T02:12:18.539 INFO:teuthology.orchestra.run.smithi133.stderr:done 2022-11-01T02:12:18.540 DEBUG:teuthology.orchestra.run.smithi133:> sudo shutdown -r now 2022-11-01T02:12:47.883 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2022-11-01T02:12:47.884 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2022-11-01T02:12:47.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-01T02:12:48.544 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2022-11-01T02:12:48.544 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2022-11-01T02:12:48.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-01T02:13:06.426 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2022-11-01T02:13:06.938 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2022-11-01T02:13:36.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2022-11-01T02:13:36.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-01T02:13:36.940 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2022-11-01T02:13:36.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-01T02:13:51.985 DEBUG:teuthology.orchestra.run.smithi012:> true 2022-11-01T02:13:52.485 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2022-11-01T02:13:52.485 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-131-generic"... 2022-11-01T02:13:52.486 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2022-11-01T02:13:52.492 INFO:teuthology.orchestra.run.smithi012.stdout:5.4.0-131-generic 2022-11-01T02:13:52.492 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-131-generic vs 5.4.0-131-generic 2022-11-01T02:13:52.492 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2022-11-01T02:13:52.493 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2022-11-01T02:13:53.494 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2022-11-01T02:13:53.495 DEBUG:teuthology.orchestra.run.smithi012:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-11-01T02:13:53.564 INFO:teuthology.orchestra.run.smithi012.stdout:ttyS1 2022-11-01T02:13:53.592 DEBUG:teuthology.parallel:result is None 2022-11-01T02:13:55.292 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2022-11-01T02:14:25.293 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2022-11-01T02:14:25.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-01T02:14:25.535 DEBUG:teuthology.orchestra.run.smithi133:> true 2022-11-01T02:14:26.093 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2022-11-01T02:14:26.094 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-131-generic"... 2022-11-01T02:14:26.094 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2022-11-01T02:14:26.101 INFO:teuthology.orchestra.run.smithi133.stdout:5.4.0-131-generic 2022-11-01T02:14:26.101 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-131-generic vs 5.4.0-131-generic 2022-11-01T02:14:26.101 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2022-11-01T02:14:26.102 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2022-11-01T02:14:27.103 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2022-11-01T02:14:27.103 DEBUG:teuthology.orchestra.run.smithi133:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-11-01T02:14:27.174 INFO:teuthology.orchestra.run.smithi133.stdout:ttyS1 2022-11-01T02:14:27.211 DEBUG:teuthology.parallel:result is None 2022-11-01T02:14:27.212 INFO:teuthology.run_tasks:Running task internal.base... 2022-11-01T02:14:27.220 INFO:teuthology.task.internal:Creating test directory... 2022-11-01T02:14:27.221 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-01T02:14:27.224 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-01T02:14:27.231 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-11-01T02:14:27.266 INFO:teuthology.run_tasks:Running task internal.archive... 2022-11-01T02:14:27.299 INFO:teuthology.task.internal:Creating archive directory... 2022-11-01T02:14:27.300 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-01T02:14:27.302 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-01T02:14:27.328 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-11-01T02:14:27.334 INFO:teuthology.task.internal:Enabling coredump saving... 2022-11-01T02:14:27.335 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-01T02:14:27.363 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-01T02:14:27.386 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-01T02:14:27.387 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-01T02:14:27.396 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-01T02:14:27.398 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-01T02:14:27.400 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-11-01T02:14:27.407 INFO:teuthology.task.internal:Configuring sudo... 2022-11-01T02:14:27.407 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-01T02:14:27.410 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-01T02:14:27.455 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-11-01T02:14:27.465 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-11-01T02:14:27.466 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-01T02:14:27.468 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-01T02:14:27.475 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-01T02:14:27.524 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-01T02:14:27.531 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2022-11-01T02:14:27.531 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-01T02:14:27.554 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-01T02:14:27.562 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-01T02:14:27.612 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2022-11-01T02:14:27.612 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-01T02:14:27.640 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2022-11-01T02:14:27.642 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2022-11-01T02:14:27.694 INFO:teuthology.run_tasks:Running task internal.timer... 2022-11-01T02:14:27.702 INFO:teuthology.task.internal:Starting timer... 2022-11-01T02:14:27.702 INFO:teuthology.run_tasks:Running task pcp... 2022-11-01T02:14:27.721 INFO:teuthology.run_tasks:Running task selinux... 2022-11-01T02:14:27.752 DEBUG:teuthology.task.selinux:Excluding smithi012: OS 'ubuntu' does not support SELinux 2022-11-01T02:14:27.752 DEBUG:teuthology.task.selinux:Excluding smithi133: OS 'ubuntu' does not support SELinux 2022-11-01T02:14:27.752 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-11-01T02:14:27.753 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2022-11-01T02:14:27.753 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-11-01T02:14:27.753 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-11-01T14:11:45.912 INFO:teuthology.repo_utils:Cloning git://git.ceph.com/git/ceph-cm-ansible.git main from upstream 2022-11-01T14:11:57.943 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2022-11-01T14:11:57.945 DEBUG:teuthology.task.console_log:Killing console logger for smithi012 2022-11-01T14:11:57.946 DEBUG:teuthology.task.console_log:Killing console logger for smithi133 2022-11-01T14:11:57.947 DEBUG:teuthology.exit:Finished running handlers