2022-07-02T09:52:06.868 INFO:root:teuthology version: 1.1.1.dev379+gf58709e3
2022-07-02T09:52:07.009 INFO:teuthology.lock.ops:Start node 'smithi080.front.sepia.ceph.com' reimaging
2022-07-02T09:52:07.010 INFO:teuthology.lock.ops:Updating [smithi080.front.sepia.ceph.com]: reset os type and version on server
2022-07-02T09:52:07.011 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server
2022-07-02T09:52:07.036 INFO:teuthology.lock.ops:Node 'smithi080.front.sepia.ceph.com' reimaging is complete
2022-07-02T09:52:07.037 INFO:teuthology.lock.ops:Start node 'smithi093.front.sepia.ceph.com' reimaging
2022-07-02T09:52:07.037 INFO:teuthology.lock.ops:Updating [smithi093.front.sepia.ceph.com]: reset os type and version on server
2022-07-02T09:52:07.038 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server
2022-07-02T09:52:07.065 INFO:teuthology.lock.ops:Node 'smithi093.front.sepia.ceph.com' reimaging is complete
2022-07-02T09:52:07.253 INFO:teuthology.provision.fog.smithi080:Scheduling deploy of rhel 8.6
2022-07-02T09:52:07.273 INFO:teuthology.provision.fog.smithi093:Scheduling deploy of rhel 8.6
2022-07-02T09:52:07.574 INFO:teuthology.orchestra.console:Power off smithi080
2022-07-02T09:52:07.612 INFO:teuthology.orchestra.console:Power off smithi093
2022-07-02T09:52:15.972 INFO:teuthology.orchestra.console:Power off for smithi093 completed
2022-07-02T09:52:16.074 INFO:teuthology.orchestra.console:Power on smithi093
2022-07-02T09:52:20.061 INFO:teuthology.orchestra.console:Power off for smithi080 completed
2022-07-02T09:52:20.163 INFO:teuthology.orchestra.console:Power on smithi080
2022-07-02T09:52:24.432 INFO:teuthology.orchestra.console:Power on for smithi093 completed
2022-07-02T09:52:24.533 INFO:teuthology.provision.fog.smithi093:Waiting for deploy to finish
2022-07-02T09:52:28.527 INFO:teuthology.orchestra.console:Power on for smithi080 completed
2022-07-02T09:52:28.628 INFO:teuthology.provision.fog.smithi080:Waiting for deploy to finish
2022-07-02T09:57:02.588 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-07-02T09:57:02.850 INFO:teuthology.provision.fog.smithi093:Node is ready
2022-07-02T09:57:02.870 INFO:teuthology.orchestra.run.smithi093.stdout:smithi093.front.sepia.ceph.com
2022-07-02T09:57:02.931 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93 smithi093.front.sepia.ceph.com smithi093
2022-07-02T09:57:03.304 INFO:teuthology.provision.fog.smithi093:Deploy complete!
2022-07-02T09:57:06.676 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-07-02T09:57:06.947 INFO:teuthology.provision.fog.smithi080:Node is ready
2022-07-02T09:57:07.007 INFO:teuthology.orchestra.run.smithi080.stdout:smithi080.front.sepia.ceph.com
2022-07-02T09:57:07.067 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80 smithi080.front.sepia.ceph.com smithi080
2022-07-02T09:57:07.430 INFO:teuthology.provision.fog.smithi080:Deploy complete!
2022-07-02T09:57:07.625 INFO:teuthology.lock.ops:Checking smithi080.front.sepia.ceph.com
2022-07-02T09:57:07.626 INFO:teuthology.lock.ops:New key found. Updating...
2022-07-02T09:57:07.648 INFO:teuthology.lock.ops:Checking smithi093.front.sepia.ceph.com
2022-07-02T09:57:07.649 INFO:teuthology.lock.ops:New key found. Updating...
2022-07-02T09:57:07.668 INFO:teuthology.lock.ops:Updating [smithi080.front.sepia.ceph.com]: set os type and version on server
2022-07-02T09:57:07.669 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com'
2022-07-02T09:57:07.877 INFO:teuthology.orchestra.run.smithi080.stdout:x86_64
2022-07-02T09:57:07.937 INFO:teuthology.orchestra.run.smithi080.stdout:NAME="Red Hat Enterprise Linux"
2022-07-02T09:57:07.937 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION="8.6 (Ootpa)"
2022-07-02T09:57:07.938 INFO:teuthology.orchestra.run.smithi080.stdout:ID="rhel"
2022-07-02T09:57:07.938 INFO:teuthology.orchestra.run.smithi080.stdout:ID_LIKE="fedora"
2022-07-02T09:57:07.938 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_ID="8.6"
2022-07-02T09:57:07.938 INFO:teuthology.orchestra.run.smithi080.stdout:PLATFORM_ID="platform:el8"
2022-07-02T09:57:07.939 INFO:teuthology.orchestra.run.smithi080.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2022-07-02T09:57:07.939 INFO:teuthology.orchestra.run.smithi080.stdout:ANSI_COLOR="0;31"
2022-07-02T09:57:07.939 INFO:teuthology.orchestra.run.smithi080.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2022-07-02T09:57:07.939 INFO:teuthology.orchestra.run.smithi080.stdout:HOME_URL="https://www.redhat.com/"
2022-07-02T09:57:07.940 INFO:teuthology.orchestra.run.smithi080.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-07-02T09:57:07.940 INFO:teuthology.orchestra.run.smithi080.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-07-02T09:57:07.940 INFO:teuthology.orchestra.run.smithi080.stdout:
2022-07-02T09:57:07.940 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-07-02T09:57:07.941 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2022-07-02T09:57:07.941 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-07-02T09:57:07.941 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2022-07-02T09:57:07.942 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server
2022-07-02T09:57:07.962 INFO:teuthology.lock.ops:Updating [smithi093.front.sepia.ceph.com]: set os type and version on server
2022-07-02T09:57:07.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com'
2022-07-02T09:57:08.170 INFO:teuthology.orchestra.run.smithi093.stdout:x86_64
2022-07-02T09:57:08.230 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="Red Hat Enterprise Linux"
2022-07-02T09:57:08.230 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="8.6 (Ootpa)"
2022-07-02T09:57:08.231 INFO:teuthology.orchestra.run.smithi093.stdout:ID="rhel"
2022-07-02T09:57:08.231 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE="fedora"
2022-07-02T09:57:08.231 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="8.6"
2022-07-02T09:57:08.231 INFO:teuthology.orchestra.run.smithi093.stdout:PLATFORM_ID="platform:el8"
2022-07-02T09:57:08.232 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2022-07-02T09:57:08.232 INFO:teuthology.orchestra.run.smithi093.stdout:ANSI_COLOR="0;31"
2022-07-02T09:57:08.232 INFO:teuthology.orchestra.run.smithi093.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2022-07-02T09:57:08.232 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://www.redhat.com/"
2022-07-02T09:57:08.232 INFO:teuthology.orchestra.run.smithi093.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-07-02T09:57:08.233 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-07-02T09:57:08.233 INFO:teuthology.orchestra.run.smithi093.stdout:
2022-07-02T09:57:08.233 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-07-02T09:57:08.233 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2022-07-02T09:57:08.234 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-07-02T09:57:08.234 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2022-07-02T09:57:08.234 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server
2022-07-02T09:57:08.335 INFO:teuthology.dispatcher.supervisor:Running job 6910336
2022-07-02T09:57:08.335 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/bin/teuthology -v --owner scheduled_nshelke@teuthology --archive /home/teuthworker/archive/nshelke-2022-07-02_08:28:15-fs:volumes-wip-nshelke-in-progress-clone-testing-distro-default-smithi/6910336 --name nshelke-2022-07-02_08:28:15-fs:volumes-wip-nshelke-in-progress-clone-testing-distro-default-smithi --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} -- /home/teuthworker/archive/nshelke-2022-07-02_08:28:15-fs:volumes-wip-nshelke-in-progress-clone-testing-distro-default-smithi/6910336/orig.config.yaml
2022-07-02T09:57:08.339 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nshelke-2022-07-02_08:28:15-fs:volumes-wip-nshelke-in-progress-clone-testing-distro-default-smithi/6910336
2022-07-02T09:57:08.340 INFO:teuthology.dispatcher.supervisor:Job PID: 429
2022-07-02T09:57:08.340 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-07-02T21:58:07.122 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-07-02T21:58:07.359 INFO:teuthology.kill:Killing Pids: {429}
2022-07-02T21:58:08.892 INFO:teuthology.task.internal:roles: ubuntu@smithi080.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2']
2022-07-02T21:58:08.893 INFO:teuthology.task.internal:roles: ubuntu@smithi093.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']
2022-07-02T21:58:08.893 INFO:teuthology.misc:Compressing logs...
2022-07-02T21:58:08.894 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com'
2022-07-02T21:58:10.470 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com'
2022-07-02T21:58:13.530 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/dispatcher/supervisor.py", line 290, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/dispatcher/supervisor.py", line 348, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/cluster.py", line 85, in run
    procs = [remote.run(**kwargs, wait=_wait) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/cluster.py", line 85, in <listcomp>
    procs = [remote.run(**kwargs, wait=_wait) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/teuthology/orchestra/remote.py", line 509, in run
    raise Exception(f'Cannot connect to remote host {self.shortname}')
Exception: Cannot connect to remote host smithi093
2022-07-02T21:58:13.611 INFO:teuthology.kill:No teuthology processes running
2022-07-02T21:58:15.121 INFO:teuthology.kill:Nuking machines: ['smithi080', 'smithi093']
2022-07-02T21:58:16.539 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_f58709e3fda190343157063e5edb0d7a36a41afc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
2022-07-02T21:58:16.539 INFO:teuthology.kill:  from cryptography.hazmat.backends import default_backend
2022-07-02T21:58:16.540 INFO:teuthology.kill:2022-07-02 21:58:16,538.538 INFO:teuthology.nuke:targets:
2022-07-02T21:58:16.540 INFO:teuthology.kill:  smithi080.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfclsHU2hFhv+1T5f9Aop1z2BqPap85PoqGf2h27N+zDFjtZ+7v/3cBRh0bVt/pJNvKSgXk+4IJ9OwX+a7ibtakvehCnf/izrt4mzhZa24LVZdLqDMc/Z2seral4EwMbQh43SmA7u0e1dJVIL989WO3L+Rw49zCjKyoh9T+Whww+rwV4dk0MRpJkQQSBZbAOI3JNM0yiXAAuvFVeTWAYJXkZ2hj/6Li/ZHy6IaIJgGHrUeZLbE/XHm+J3Mv6hAsH6dQRlr4EObXOyerLFuuJhq9uhSgjKy3tIOx3A9hGhTXcyDW9kx/wNPQH8yMdOB4YlXwFHEiUJHZwFpDZEw/VSQw8CNjmzSaAlqg9U9DE3nPm0FNZ8S62rYxJOetkx+aUayMyrWG/niqlc6yZGDWZsUYLTkGPwxLeqcNCBlZwpaPRwWdYjXBz/uuM1ub3R1pGRiSkZ9894SrvXPlFpzJ0ntCokh3u+izZpNhPoJGaNBr82b0PuJBvODjYQNsvLlHl8=
2022-07-02T21:58:16.541 INFO:teuthology.kill:  smithi093.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCukxbNha6hB8chhVTe0F6eihIKO6Y2eCCFA2+WLKxLnkuCWCQPp2m2MQVjzq+frqWJAtgNY7+/xYymJUbb2J7FyJ5DeMtRvYLpJCgE8DS+tVY4OXN8tAZNj0IjO4LIfXxw9RpbhR222Xv2wZtXwMlIAeLWoSiSpVAlpHj1z7N/rQlxJJ1Bogq9qWcpCxazhp42QMmlOLRJAAetLDL3hESDTi+acD1gfiyvA3Dww9YM1yf9ElQcie9IeOO6pG6qplL6ONaYvOgm3ujsz5UG+bGlx9gb2Gwh6m+ll5T+i2kWkX5zlqnLD+QUi2WntjfQkX9zPastQYZnFKnPnkBvMQesqizwfPXv6M0XjWjrOqURozPje40xebrUiAUe+t6B0UYEP3o09b/PBh1QiduMv3ApvTkMpIxqbYA+7G2xW9oAAJmPSnKmbW7LDv3apRFWn1/8wpZcgA+rM+F4Id2wUkI+rJC079pCsPwQQfQalHZcTy8yWsm3v3Ny37/pBjNrTbU=
2022-07-02T21:58:16.541 INFO:teuthology.kill:2022-07-02 21:58:16,538.538 INFO:teuthology.nuke:Checking targets against current locks
2022-07-02T21:58:16.599 INFO:teuthology.kill:2022-07-02 21:58:16,599.599 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-07-02T21:58:16.619 INFO:teuthology.kill:2022-07-02 21:58:16,619.619 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-07-02T21:58:16.684 INFO:teuthology.kill:2022-07-02 21:58:16,684.684 INFO:teuthology.orchestra.console:Power off smithi080
2022-07-02T21:58:16.811 INFO:teuthology.kill:2022-07-02 21:58:16,811.811 INFO:teuthology.orchestra.console:Power off smithi093
2022-07-02T21:58:25.051 INFO:teuthology.kill:2022-07-02 21:58:25,051.051 INFO:teuthology.orchestra.console:Power off for smithi080 completed
2022-07-02T21:58:25.209 INFO:teuthology.kill:2022-07-02 21:58:25,207.207 INFO:teuthology.lock.ops:unlocked: smithi080.front.sepia.ceph.com
2022-07-02T21:58:29.290 INFO:teuthology.kill:2022-07-02 21:58:29,290.290 INFO:teuthology.orchestra.console:Power off for smithi093 completed
2022-07-02T21:58:29.442 INFO:teuthology.kill:2022-07-02 21:58:29,442.442 INFO:teuthology.lock.ops:unlocked: smithi093.front.sepia.ceph.com
2022-07-02T22:00:29.872 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-07-02T22:00:30.001 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi080 but it was locked by another job: /home/teuthworker/archive/rishabh-2022-07-02_14:14:52-fs-wip-rishabh-testing-20220702-1631-testing-default-smithi/6910585
2022-07-02T22:00:30.002 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi093 but it was locked by another job: /home/teuthworker/archive/rishabh-2022-07-02_14:14:52-fs-wip-rishabh-testing-20220702-1631-testing-default-smithi/6910585