2022-07-02T17:01:19.062 INFO:root:teuthology version: 1.1.1.dev379+gf58709e3 2022-07-02T17:01:19.216 INFO:teuthology.lock.ops:Start node 'smithi081.front.sepia.ceph.com' reimaging 2022-07-02T17:01:19.217 INFO:teuthology.lock.ops:Updating [smithi081.front.sepia.ceph.com]: reset os type and version on server 2022-07-02T17:01:19.217 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2022-07-02T17:01:19.238 INFO:teuthology.lock.ops:Node 'smithi081.front.sepia.ceph.com' reimaging is complete 2022-07-02T17:01:19.238 INFO:teuthology.lock.ops:Start node 'smithi174.front.sepia.ceph.com' reimaging 2022-07-02T17:01:19.239 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: reset os type and version on server 2022-07-02T17:01:19.239 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2022-07-02T17:01:19.260 INFO:teuthology.lock.ops:Node 'smithi174.front.sepia.ceph.com' reimaging is complete 2022-07-02T17:01:19.447 INFO:teuthology.provision.fog.smithi081:Scheduling deploy of ubuntu 20.04 2022-07-02T17:01:19.470 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of ubuntu 20.04 2022-07-02T17:01:19.749 INFO:teuthology.orchestra.console:Power off smithi081 2022-07-02T17:01:19.768 INFO:teuthology.orchestra.console:Power off smithi174 2022-07-02T17:01:28.137 INFO:teuthology.orchestra.console:Power off for smithi174 completed 2022-07-02T17:01:28.238 INFO:teuthology.orchestra.console:Power on smithi174 2022-07-02T17:01:32.258 INFO:teuthology.orchestra.console:Power off for smithi081 completed 2022-07-02T17:01:32.360 INFO:teuthology.orchestra.console:Power on smithi081 2022-07-02T17:01:36.613 INFO:teuthology.orchestra.console:Power on for smithi081 completed 2022-07-02T17:01:36.614 INFO:teuthology.orchestra.console:Power on for smithi174 completed 2022-07-02T17:01:36.715 INFO:teuthology.provision.fog.smithi081:Waiting for deploy to finish 2022-07-02T17:01:36.717 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish 2022-07-02T17:04:22.947 ERROR:teuthology.orchestra.connection:Error authenticating with smithi174.front.sepia.ceph.com: Authentication failed. 2022-07-02T17:06:25.635 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-02T17:06:26.157 INFO:teuthology.provision.fog.smithi174:Node is ready 2022-07-02T17:06:26.164 INFO:teuthology.orchestra.run.smithi174.stdout:smithi174.front.sepia.ceph.com 2022-07-02T17:06:26.173 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.174 smithi174.front.sepia.ceph.com smithi174 2022-07-02T17:06:26.365 INFO:teuthology.provision.fog.smithi174:Deploy complete! 2022-07-02T17:06:35.721 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-02T17:06:36.215 INFO:teuthology.provision.fog.smithi081:Node is ready 2022-07-02T17:06:36.229 INFO:teuthology.orchestra.run.smithi081.stdout:smithi081.front.sepia.ceph.com 2022-07-02T17:06:36.281 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81 smithi081.front.sepia.ceph.com smithi081 2022-07-02T17:06:36.446 INFO:teuthology.provision.fog.smithi081:Deploy complete! 2022-07-02T17:06:36.763 INFO:teuthology.lock.ops:Checking smithi081.front.sepia.ceph.com 2022-07-02T17:06:36.764 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-02T17:06:36.786 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com 2022-07-02T17:06:36.787 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-02T17:06:36.811 INFO:teuthology.lock.ops:Updating [smithi081.front.sepia.ceph.com]: set os type and version on server 2022-07-02T17:06:36.812 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2022-07-02T17:06:37.043 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2022-07-02T17:06:37.053 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="Ubuntu" 2022-07-02T17:06:37.054 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-07-02T17:06:37.054 INFO:teuthology.orchestra.run.smithi081.stdout:ID=ubuntu 2022-07-02T17:06:37.055 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE=debian 2022-07-02T17:06:37.055 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-07-02T17:06:37.056 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="20.04" 2022-07-02T17:06:37.056 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://www.ubuntu.com/" 2022-07-02T17:06:37.057 INFO:teuthology.orchestra.run.smithi081.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-07-02T17:06:37.057 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-07-02T17:06:37.058 INFO:teuthology.orchestra.run.smithi081.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-07-02T17:06:37.058 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_CODENAME=focal 2022-07-02T17:06:37.059 INFO:teuthology.orchestra.run.smithi081.stdout:UBUNTU_CODENAME=focal 2022-07-02T17:06:37.062 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2022-07-02T17:06:37.085 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server 2022-07-02T17:06:37.086 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2022-07-02T17:06:37.323 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2022-07-02T17:06:37.331 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="Ubuntu" 2022-07-02T17:06:37.331 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-07-02T17:06:37.331 INFO:teuthology.orchestra.run.smithi174.stdout:ID=ubuntu 2022-07-02T17:06:37.332 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE=debian 2022-07-02T17:06:37.332 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-07-02T17:06:37.332 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="20.04" 2022-07-02T17:06:37.332 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://www.ubuntu.com/" 2022-07-02T17:06:37.332 INFO:teuthology.orchestra.run.smithi174.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-07-02T17:06:37.333 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-07-02T17:06:37.333 INFO:teuthology.orchestra.run.smithi174.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-07-02T17:06:37.333 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_CODENAME=focal 2022-07-02T17:06:37.333 INFO:teuthology.orchestra.run.smithi174.stdout:UBUNTU_CODENAME=focal 2022-07-02T17:06:37.334 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2022-07-02T17:06:37.464 INFO:teuthology.dispatcher.supervisor:Running job 6909637 2022-07-02T17:06:37.465 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2022-07-01_20:15:02-fs-wip-adk-testing-2022-06-30-2110-distro-default-smithi/6909637 --name adking-2022-07-01_20:15:02-fs-wip-adk-testing-2022-06-30-2110-distro-default-smithi --description fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} -- /home/teuthworker/archive/adking-2022-07-01_20:15:02-fs-wip-adk-testing-2022-06-30-2110-distro-default-smithi/6909637/orig.config.yaml 2022-07-02T17:06:37.472 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2022-07-01_20:15:02-fs-wip-adk-testing-2022-06-30-2110-distro-default-smithi/6909637 2022-07-02T17:06:37.473 INFO:teuthology.dispatcher.supervisor:Job PID: 30180 2022-07-02T17:06:37.473 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-07-03T05:07:03.744 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-07-03T05:07:03.923 INFO:teuthology.kill:Killing Pids: {30180} 2022-07-03T05:07:05.325 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-07-03T05:07:05.326 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2022-07-03T05:07:05.327 INFO:teuthology.misc:Compressing logs... 2022-07-03T05:07:05.327 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2022-07-03T05:07:07.034 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/dispatcher/supervisor.py", line 290, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/dispatcher/supervisor.py", line 348, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/remote.py", line 509, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi081 2022-07-03T05:07:07.107 INFO:teuthology.kill:No teuthology processes running 2022-07-03T05:07:08.475 INFO:teuthology.kill:Nuking machines: ['smithi081', 'smithi174'] 2022-07-03T05:07:09.680 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-07-03T05:07:09.681 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-07-03T05:07:09.681 INFO:teuthology.kill:2022-07-03 05:07:09,679.679 INFO:teuthology.nuke:targets: 2022-07-03T05:07:09.682 INFO:teuthology.kill: smithi081.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCd9mWJE13xhsddEHpie/rMbrmnc/u+b0iVukce/3XA7FjSsO+Semwcg/9QkrX5IzeaFIJuV/OPNP0jYkpCwzmuhAMq4jpdgSsVa6zIxmZhrTgL4CYT3zoc2qiA2erXv0xKCflWRCMbFKkpCPn832uchL5pgb9bvKlpjyCmEBZOteEKuE5UWg71hMzRaG8MDVkHtEw1zVAuqkDe/rQ7CbaY52PuUf62UUpZmXdxCvFLmFER2skLSRZnHhUD6xjsZAuzpNJlLxds+93TvlWttERt5QjH9SV2Ps8D4Jk5T1eSSD7irlA6zGXq9pHX6nFJV54T6YUffzZ+uGD5rEJ4ZkBO9pVgvIi4wSCI9h1c0ewCLl7xuXPk4DiNwap8jPClvgFmLlIxEWgN257OxYomyWcdT3YQkwSYPs0v+al4c3Nh+TzDQ5ztWvi6ETIwkPTWhBlkKzIj47pnH/46GXviQjK5XrjRwyca8HjUGn5XEV4Jb2Rqg90ipu6M3cjruLG2Oqs= 2022-07-03T05:07:09.682 INFO:teuthology.kill: smithi174.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbYsOfBBMWhfG6/HK0z4IEGthvkCBiF8A7wui1OvG8UWEza7K8YyjJKI8IAh/nUQu36D+uhKAI00sCH/uXOj6WIPIkuDHBSKlVFzszySrM26a5QtgiCuFH+phLoSVrZTxLeHfzrFupwhe8FsFkyDxy1d2ApE4O+vKlpCYAw+YQ86MA7LN4IahmZu/rIgR5pkmDEXBv9p/xqYkl4JSEB462d8Bfp1yGdHr0EfM5kCltewHu7TUSkOyi+81zGADiagJLVDb+7JYczDcR0r2nY3as45gu+KRQJlf1aPJrkjS2/TJu6zhBCXgehNBpFVq+jHjrzQMr7jtOOM9cWk7dy/4UtfQg2TdMDwm2w4djwSR3gctB4yRqSV7ROYbGz742tjRj8UN8pIbdxZ/mtebE5jaaxmGXO/A2VhSsZDkK+gQt9fN7IejHawibPQx+Xkbayz1x03e/AnBwj8ZYtuS3H5mXf2La9q1bAlkXhofIfYWzVgwcJCJPg7L0DDaLithxeBU= 2022-07-03T05:07:09.683 INFO:teuthology.kill:2022-07-03 05:07:09,680.680 INFO:teuthology.nuke:Checking targets against current locks 2022-07-03T05:07:09.747 INFO:teuthology.kill:2022-07-03 05:07:09,747.747 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-03T05:07:09.777 INFO:teuthology.kill:2022-07-03 05:07:09,777.777 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-03T05:07:09.849 INFO:teuthology.kill:2022-07-03 05:07:09,849.849 INFO:teuthology.orchestra.console:Power off smithi081 2022-07-03T05:07:09.890 INFO:teuthology.kill:2022-07-03 05:07:09,889.889 INFO:teuthology.orchestra.console:Power off smithi174 2022-07-03T05:07:18.221 INFO:teuthology.kill:2022-07-03 05:07:18,220.220 INFO:teuthology.orchestra.console:Power off for smithi081 completed 2022-07-03T05:07:18.255 INFO:teuthology.kill:2022-07-03 05:07:18,255.255 INFO:teuthology.orchestra.console:Power off for smithi174 completed 2022-07-03T05:07:18.377 INFO:teuthology.kill:2022-07-03 05:07:18,376.376 INFO:teuthology.lock.ops:unlocked: smithi081.front.sepia.ceph.com 2022-07-03T05:07:18.407 INFO:teuthology.kill:2022-07-03 05:07:18,407.407 INFO:teuthology.lock.ops:unlocked: smithi174.front.sepia.ceph.com 2022-07-03T05:09:18.652 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-07-03T05:09:18.796 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi081 but it was locked by another job: /home/teuthworker/archive/rishabh-2022-07-02_14:14:52-fs-wip-rishabh-testing-20220702-1631-testing-default-smithi/6910783 2022-07-03T05:09:18.797 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi174 but it was locked by another job: /home/teuthworker/archive/rishabh-2022-07-02_14:14:52-fs-wip-rishabh-testing-20220702-1631-testing-default-smithi/6910783