2022-07-03T08:52:29.150 INFO:root:teuthology version: 1.1.1.dev379+gf58709e3 2022-07-03T08:52:29.287 INFO:teuthology.lock.ops:Start node 'smithi153.front.sepia.ceph.com' reimaging 2022-07-03T08:52:29.288 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: reset os type and version on server 2022-07-03T08:52:29.288 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2022-07-03T08:52:29.309 INFO:teuthology.lock.ops:Node 'smithi153.front.sepia.ceph.com' reimaging is complete 2022-07-03T08:52:29.310 INFO:teuthology.lock.ops:Start node 'smithi197.front.sepia.ceph.com' reimaging 2022-07-03T08:52:29.312 INFO:teuthology.lock.ops:Updating [smithi197.front.sepia.ceph.com]: reset os type and version on server 2022-07-03T08:52:29.313 INFO:teuthology.lock.ops:Updating smithi197.front.sepia.ceph.com on lock server 2022-07-03T08:52:29.333 INFO:teuthology.lock.ops:Node 'smithi197.front.sepia.ceph.com' reimaging is complete 2022-07-03T08:52:29.527 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of ubuntu 20.04 2022-07-03T08:52:29.540 INFO:teuthology.provision.fog.smithi197:Scheduling deploy of ubuntu 20.04 2022-07-03T08:52:29.825 INFO:teuthology.orchestra.console:Power off smithi153 2022-07-03T08:52:29.839 INFO:teuthology.orchestra.console:Power off smithi197 2022-07-03T08:52:42.313 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2022-07-03T08:52:42.321 INFO:teuthology.orchestra.console:Power off for smithi197 completed 2022-07-03T08:52:42.415 INFO:teuthology.orchestra.console:Power on smithi153 2022-07-03T08:52:42.424 INFO:teuthology.orchestra.console:Power on smithi197 2022-07-03T08:52:46.680 INFO:teuthology.orchestra.console:Power on for smithi153 completed 2022-07-03T08:52:46.685 INFO:teuthology.orchestra.console:Power on for smithi197 completed 2022-07-03T08:52:46.781 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish 2022-07-03T08:52:46.786 INFO:teuthology.provision.fog.smithi197:Waiting for deploy to finish 2022-07-03T08:57:47.411 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-03T08:57:48.024 INFO:teuthology.provision.fog.smithi197:Node is ready 2022-07-03T08:57:48.030 INFO:teuthology.orchestra.run.smithi197.stdout:smithi197.front.sepia.ceph.com 2022-07-03T08:57:48.079 INFO:teuthology.orchestra.run.smithi197.stdout:172.21.15.197 smithi197.front.sepia.ceph.com smithi197 2022-07-03T08:57:48.266 INFO:teuthology.provision.fog.smithi197:Deploy complete! 2022-07-03T08:57:53.396 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-03T08:57:53.939 INFO:teuthology.provision.fog.smithi153:Node is ready 2022-07-03T08:57:53.945 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com 2022-07-03T08:57:53.953 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153 2022-07-03T08:57:54.174 INFO:teuthology.provision.fog.smithi153:Deploy complete! 2022-07-03T08:57:54.458 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com 2022-07-03T08:57:54.459 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-03T08:57:54.496 INFO:teuthology.lock.ops:Checking smithi197.front.sepia.ceph.com 2022-07-03T08:57:54.496 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-03T08:57:54.524 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server 2022-07-03T08:57:54.525 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2022-07-03T08:57:54.757 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2022-07-03T08:57:54.764 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="Ubuntu" 2022-07-03T08:57:54.765 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-07-03T08:57:54.765 INFO:teuthology.orchestra.run.smithi153.stdout:ID=ubuntu 2022-07-03T08:57:54.766 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE=debian 2022-07-03T08:57:54.766 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-07-03T08:57:54.767 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="20.04" 2022-07-03T08:57:54.767 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://www.ubuntu.com/" 2022-07-03T08:57:54.767 INFO:teuthology.orchestra.run.smithi153.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-07-03T08:57:54.768 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-07-03T08:57:54.768 INFO:teuthology.orchestra.run.smithi153.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-07-03T08:57:54.769 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_CODENAME=focal 2022-07-03T08:57:54.769 INFO:teuthology.orchestra.run.smithi153.stdout:UBUNTU_CODENAME=focal 2022-07-03T08:57:54.771 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2022-07-03T08:57:54.795 INFO:teuthology.lock.ops:Updating [smithi197.front.sepia.ceph.com]: set os type and version on server 2022-07-03T08:57:54.795 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi197.front.sepia.ceph.com' 2022-07-03T08:57:55.023 INFO:teuthology.orchestra.run.smithi197.stdout:x86_64 2022-07-03T08:57:55.029 INFO:teuthology.orchestra.run.smithi197.stdout:NAME="Ubuntu" 2022-07-03T08:57:55.029 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-07-03T08:57:55.030 INFO:teuthology.orchestra.run.smithi197.stdout:ID=ubuntu 2022-07-03T08:57:55.030 INFO:teuthology.orchestra.run.smithi197.stdout:ID_LIKE=debian 2022-07-03T08:57:55.030 INFO:teuthology.orchestra.run.smithi197.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-07-03T08:57:55.031 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION_ID="20.04" 2022-07-03T08:57:55.031 INFO:teuthology.orchestra.run.smithi197.stdout:HOME_URL="https://www.ubuntu.com/" 2022-07-03T08:57:55.031 INFO:teuthology.orchestra.run.smithi197.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-07-03T08:57:55.032 INFO:teuthology.orchestra.run.smithi197.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-07-03T08:57:55.032 INFO:teuthology.orchestra.run.smithi197.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-07-03T08:57:55.033 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION_CODENAME=focal 2022-07-03T08:57:55.033 INFO:teuthology.orchestra.run.smithi197.stdout:UBUNTU_CODENAME=focal 2022-07-03T08:57:55.033 INFO:teuthology.lock.ops:Updating smithi197.front.sepia.ceph.com on lock server 2022-07-03T08:57:55.235 INFO:teuthology.dispatcher.supervisor:Running job 6910824 2022-07-03T08:57:55.236 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2022-07-02_14:14:52-fs-wip-rishabh-testing-20220702-1631-testing-default-smithi/6910824 --name rishabh-2022-07-02_14:14:52-fs-wip-rishabh-testing-20220702-1631-testing-default-smithi --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} -- /home/teuthworker/archive/rishabh-2022-07-02_14:14:52-fs-wip-rishabh-testing-20220702-1631-testing-default-smithi/6910824/orig.config.yaml 2022-07-03T08:57:55.241 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2022-07-02_14:14:52-fs-wip-rishabh-testing-20220702-1631-testing-default-smithi/6910824 2022-07-03T08:57:55.242 INFO:teuthology.dispatcher.supervisor:Job PID: 23889 2022-07-03T08:57:55.242 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-07-03T20:58:53.332 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-07-03T20:58:53.515 INFO:teuthology.kill:Killing Pids: {23889} 2022-07-03T20:58:55.071 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2022-07-03T20:58:55.072 INFO:teuthology.task.internal:roles: ubuntu@smithi197.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2022-07-03T20:58:55.072 INFO:teuthology.misc:Compressing logs... 2022-07-03T20:58:55.073 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2022-07-03T20:58:56.634 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/dispatcher/supervisor.py", line 290, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/dispatcher/supervisor.py", line 348, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/remote.py", line 509, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi153 2022-07-03T20:58:56.703 INFO:teuthology.kill:No teuthology processes running 2022-07-03T20:58:58.109 INFO:teuthology.kill:Nuking machines: ['smithi153', 'smithi197'] 2022-07-03T20:58:59.305 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-07-03T20:58:59.306 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-07-03T20:58:59.306 INFO:teuthology.kill:2022-07-03 20:58:59,304.304 INFO:teuthology.nuke:targets: 2022-07-03T20:58:59.307 INFO:teuthology.kill: smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUi4Ry0ZgTbpTKV65NEEehpVn36jM13En059lq7vegr6noJmQ/WFsMQd0yUU5tqVz/xq+KtFbIN33siWaUjEBXBdPT4h/cPHurCvohKGox5n27Qmuz579xMmQnBqrhhZdpkeJOpz0zkW/hZEvcNVzdhyWBz11FTStL5gDhRD7PJqsDVUhDMmKy43RTYajiEIYEgqfYbWRssKJsMEh6+25r2+zmRrsfsJUNF+D0VEyIGsXmgXN9R3gnYd5nXb/1BRdymn1S0jZAPB7prg7JyfN/GNmKl/OfBhXBxWyyG69Zn3IwVyoM90+ptxpNv55dby/uMMAg+akGoJxKlZOvO4cZ1cVQy3RNADf8k1x0uPCnkDiXI06E9k0sUg8/OtZS/u7tkqeILNFL2WaZguVzdCFbCH2k00rYxJ6wm4PQFCPRMo45eXpUCDCKaMx1Cu0tdTYRMJSqx01kO4/28oeIwIR494TRrUfWoN5nwr2kX+UM0n0MsueFt+kBVS3ucholFjc= 2022-07-03T20:58:59.307 INFO:teuthology.kill: smithi197.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8T0ZGJg9/i5z12aI3pLlZkv3Km65o9yLXjkKEeFPB/Rtt+oN/MxpaspUnFCkJ+M0bWqEzJRDFitJIG/psenfHKLGPXNEBD91QdsJd93p2D5cXB4L9Cs9Fz0SyLNlovU0bZ4aTk5JVI25RuJDr3drkOFdz4HhoEcSPG8B8Q3cDNpJSm1JZh0YheEz2INRTlVUxZ5zT9jiEYl33Mc3krD1b1EFUyTv2IR4lxoTEnzIfaxVlRcBTWsEI/WJYLrcJ/t1bV8ymNezMx9HsXoRDJeD/erUeprS0bgbnFwRCtxwDEuneFav8hBTYtTorZ0Toy3EdMlPxON8gjmxZyUk2noTyCQLepUGaVvIuIy0Z9kLEcqJSLJxS1XlMeGLrq35Z/X2n/R6hn54sq1Y/4IqqD34aFsMYvrXNBgrRC2TUzraieetQlj0xwseTb6XLLFwpyzdmmGPLFKjYv3UT/SyeZdUGxo70f7RZbLKjSLjM1UopkfDYc0PSxAMJTiE/FhHRM9k= 2022-07-03T20:58:59.307 INFO:teuthology.kill:2022-07-03 20:58:59,305.305 INFO:teuthology.nuke:Checking targets against current locks 2022-07-03T20:58:59.362 INFO:teuthology.kill:2022-07-03 20:58:59,362.362 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-03T20:58:59.390 INFO:teuthology.kill:2022-07-03 20:58:59,389.389 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-03T20:58:59.450 INFO:teuthology.kill:2022-07-03 20:58:59,450.450 INFO:teuthology.orchestra.console:Power off smithi153 2022-07-03T20:58:59.491 INFO:teuthology.kill:2022-07-03 20:58:59,491.491 INFO:teuthology.orchestra.console:Power off smithi197 2022-07-03T20:59:07.828 INFO:teuthology.kill:2022-07-03 20:59:07,827.827 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2022-07-03T20:59:07.857 INFO:teuthology.kill:2022-07-03 20:59:07,857.857 INFO:teuthology.orchestra.console:Power off for smithi197 completed 2022-07-03T20:59:07.978 INFO:teuthology.kill:2022-07-03 20:59:07,977.977 INFO:teuthology.lock.ops:unlocked: smithi153.front.sepia.ceph.com 2022-07-03T20:59:08.007 INFO:teuthology.kill:2022-07-03 20:59:08,007.007 INFO:teuthology.lock.ops:unlocked: smithi197.front.sepia.ceph.com 2022-07-03T21:01:08.320 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15