2021-04-08T10:18:16.588 INFO:root:teuthology version: 1.1.0-85d61eae
2021-04-08T10:18:16.710 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi035.front.sepia.ceph.com' reimaging
2021-04-08T10:18:16.711 INFO:teuthology.lock.ops:Updating [ubuntu@smithi035.front.sepia.ceph.com]: reset os type and version on server
2021-04-08T10:18:16.711 INFO:teuthology.lock.ops:Updating smithi035.front.sepia.ceph.com on lock server
2021-04-08T10:18:16.740 INFO:teuthology.lock.ops:Node 'ubuntu@smithi035.front.sepia.ceph.com' reimaging is complete
2021-04-08T10:18:16.740 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi066.front.sepia.ceph.com' reimaging
2021-04-08T10:18:16.741 INFO:teuthology.lock.ops:Updating [ubuntu@smithi066.front.sepia.ceph.com]: reset os type and version on server
2021-04-08T10:18:16.741 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server
2021-04-08T10:18:16.764 INFO:teuthology.lock.ops:Node 'ubuntu@smithi066.front.sepia.ceph.com' reimaging is complete
2021-04-08T10:18:16.990 INFO:teuthology.provision.fog.smithi035:Scheduling deploy of centos 8.3
2021-04-08T10:18:17.033 INFO:teuthology.provision.fog.smithi066:Scheduling deploy of centos 8.3
2021-04-08T10:18:17.502 INFO:teuthology.orchestra.console:Power off smithi066
2021-04-08T10:18:17.599 INFO:teuthology.orchestra.console:Power off smithi035
2021-04-08T10:18:30.022 INFO:teuthology.orchestra.console:Power off for smithi066 completed
2021-04-08T10:18:30.086 INFO:teuthology.orchestra.console:Power off for smithi035 completed
2021-04-08T10:18:30.124 INFO:teuthology.orchestra.console:Power on smithi066
2021-04-08T10:18:30.188 INFO:teuthology.orchestra.console:Power on smithi035
2021-04-08T10:18:34.380 INFO:teuthology.orchestra.console:Power on for smithi066 completed
2021-04-08T10:18:34.457 INFO:teuthology.orchestra.console:Power on for smithi035 completed
2021-04-08T10:18:34.481 INFO:teuthology.provision.fog.smithi066:Waiting for deploy to finish
2021-04-08T10:18:34.559 INFO:teuthology.provision.fog.smithi035:Waiting for deploy to finish
2021-04-08T10:20:50.742 ERROR:teuthology.orchestra.connection:Error authenticating with smithi035.front.sepia.ceph.com: Authentication failed.
2021-04-08T10:22:22.515 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-04-08T10:22:37.635 INFO:teuthology.provision.fog.smithi035:Node is ready
2021-04-08T10:22:37.659 INFO:teuthology.orchestra.run.smithi035.stdout:smithi035.front.sepia.ceph.com
2021-04-08T10:22:37.716 INFO:teuthology.orchestra.run.smithi035.stdout:172.21.15.35 smithi035.front.sepia.ceph.com smithi035
2021-04-08T10:22:38.065 INFO:teuthology.provision.fog.smithi035:Deploy complete!
2021-04-08T10:22:43.991 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-04-08T10:23:04.779 INFO:teuthology.provision.fog.smithi066:Node is ready
2021-04-08T10:23:04.800 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066.front.sepia.ceph.com
2021-04-08T10:23:04.860 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66 smithi066.front.sepia.ceph.com smithi066
2021-04-08T10:23:05.232 INFO:teuthology.provision.fog.smithi066:Deploy complete!
2021-04-08T10:23:05.404 INFO:teuthology.lock.ops:Checking smithi035.front.sepia.ceph.com
2021-04-08T10:23:05.410 INFO:teuthology.lock.ops:New key found. Updating...
2021-04-08T10:23:05.435 INFO:teuthology.lock.ops:Checking smithi066.front.sepia.ceph.com
2021-04-08T10:23:05.436 INFO:teuthology.lock.ops:New key found. Updating...
2021-04-08T10:23:05.479 INFO:teuthology.lock.ops:Updating [smithi035.front.sepia.ceph.com]: set os type and version on server
2021-04-08T10:23:05.480 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-04-08T10:23:05.782 INFO:teuthology.orchestra.run.smithi035.stdout:x86_64
2021-04-08T10:23:05.838 INFO:teuthology.orchestra.run.smithi035.stdout:NAME="CentOS Linux"
2021-04-08T10:23:05.838 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION="8"
2021-04-08T10:23:05.839 INFO:teuthology.orchestra.run.smithi035.stdout:ID="centos"
2021-04-08T10:23:05.839 INFO:teuthology.orchestra.run.smithi035.stdout:ID_LIKE="rhel fedora"
2021-04-08T10:23:05.840 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION_ID="8"
2021-04-08T10:23:05.840 INFO:teuthology.orchestra.run.smithi035.stdout:PLATFORM_ID="platform:el8"
2021-04-08T10:23:05.840 INFO:teuthology.orchestra.run.smithi035.stdout:PRETTY_NAME="CentOS Linux 8"
2021-04-08T10:23:05.841 INFO:teuthology.orchestra.run.smithi035.stdout:ANSI_COLOR="0;31"
2021-04-08T10:23:05.841 INFO:teuthology.orchestra.run.smithi035.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-04-08T10:23:05.842 INFO:teuthology.orchestra.run.smithi035.stdout:HOME_URL="https://centos.org/"
2021-04-08T10:23:05.842 INFO:teuthology.orchestra.run.smithi035.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-04-08T10:23:05.843 INFO:teuthology.orchestra.run.smithi035.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-04-08T10:23:05.843 INFO:teuthology.orchestra.run.smithi035.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-04-08T10:23:05.845 INFO:teuthology.lock.ops:Updating smithi035.front.sepia.ceph.com on lock server
2021-04-08T10:23:05.864 INFO:teuthology.lock.ops:Updating [smithi066.front.sepia.ceph.com]: set os type and version on server
2021-04-08T10:23:05.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-04-08T10:23:06.136 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64
2021-04-08T10:23:06.192 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Linux"
2021-04-08T10:23:06.193 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="8"
2021-04-08T10:23:06.193 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos"
2021-04-08T10:23:06.193 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora"
2021-04-08T10:23:06.194 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="8"
2021-04-08T10:23:06.194 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el8"
2021-04-08T10:23:06.194 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Linux 8"
2021-04-08T10:23:06.194 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31"
2021-04-08T10:23:06.194 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-04-08T10:23:06.195 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/"
2021-04-08T10:23:06.195 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-04-08T10:23:06.195 INFO:teuthology.orchestra.run.smithi066.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-04-08T10:23:06.196 INFO:teuthology.orchestra.run.smithi066.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-04-08T10:23:06.196 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server
2021-04-08T10:23:06.284 INFO:teuthology.dispatcher.supervisor:Running job 6028704
2021-04-08T10:23:06.285 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/bin/teuthology -v --owner scheduled_rzarzynski@teuthology --archive /home/teuthworker/archive/rzarzynski-2021-04-08_10:14:11-rados-master-distro-basic-smithi/6028704 --name rzarzynski-2021-04-08_10:14:11-rados-master-distro-basic-smithi --description rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/rados_5925} -- /home/teuthworker/archive/rzarzynski-2021-04-08_10:14:11-rados-master-distro-basic-smithi/6028704/orig.config.yaml
2021-04-08T10:23:06.289 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rzarzynski-2021-04-08_10:14:11-rados-master-distro-basic-smithi/6028704
2021-04-08T10:23:06.291 INFO:teuthology.dispatcher.supervisor:Job PID: 12523
2021-04-08T10:23:06.292 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-04-08T22:23:14.605 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-04-08T22:23:14.798 INFO:teuthology.kill:Killing Pids: {12523}
2021-04-08T22:23:16.045 INFO:teuthology.kill:Nuking machines: ['smithi035', 'smithi066']
2021-04-08T22:23:17.197 INFO:teuthology.kill:2021-04-08 22:23:17,196.196 INFO:teuthology.nuke:targets:
2021-04-08T22:23:17.198 INFO:teuthology.kill:  smithi035.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDK93THBCon6CuiF99H1IPX/ZEBZ23tlpzEiu/2V9aUUkxlrRwPQFWd2lvAqE4T2aXGmAE4p9ldr15ZRd7DJgEONsvjlH1rBbga7B3Klf69kcfPAY+/y9lUP2WdY6E1Yp1j87Z5dW4meLI/mRZdkkCP6nJZZ9P98sb2mZHV8Kt71G+dGFx1d+HS6Vcw8Bk7eOimqhvGYW7uE1nGK3VJPqdZ2A5oFhWcazG9fQspbVTxhvNQQEUA3WUmQn6Tw5ZXjBSsweQH4xxvAYEuQkU5m6IQ1bbOrl+xRsSl0FW7+n+ewBc7lqG0HZrYNPqjVnyIlTDTOHkmhDUSIK9u0/k3AzNh4a3iLLbszTN+WZLa5RXPh15uudpuSOfl+Tttzt6PNfZgNgeQp342RRd06uaaqPJPlS1nCFTqH+AmrqePK0nKP74ONMT9UH1Z68KuQ6N+kYOplLWWACKPRUYo5C2op8f+mSzGFH9Trp58k7PGPzGuzc0P/RTTsCdrkWfTway5GT0=
2021-04-08T22:23:17.198 INFO:teuthology.kill:  smithi066.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIpwfnwSi8MFysCVt9RGpHY8OCWJy46pghnS/f3iGpRC0tHI/MuXS1Rv4ZHxjOMkXSY1KcpgBHQ4B6R4cDFpvuxBvBmiKYpaasBWX5FkPtcAd6PPBKx0nFU0JTuU59ERRkBWtzwZUVn5u7Kp3KdjOtsfKqKjgydVTApYs70Yx6pdbj/Yt02fzyol98H6sD9EqYgA9GhBn4mq26/2M+kM67/e70WgyLFdsTS/vAM5BYVx70yPKatyYFiVJaEVF5gHclR2N7PpTaW4VsGyP3Tc40unHE/YnXv8NGSd4eX1VnN89uyeqx11gwwNfy9SLk4+vl3UoCo8CxddRrA8PepOP5LhRTkmCzZJvRRA1jYjegjcVVI8fZGPXRHYQSGsk26L+Z9VI1nKNarXTQ3JVOvvSuC6ZmAU704TkOXgiCGLXoTnX74VqY7zfhY+BEC7hxXfQNphGXjDR1xqqZRcbZDiYq4e+egorWCITs/L8tJy5kvaQyOmeOB/DWD3UzIqtnuEc=
2021-04-08T22:23:17.198 INFO:teuthology.kill:2021-04-08 22:23:17,197.197 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-08T22:23:17.199 INFO:teuthology.kill:2021-04-08 22:23:17,199.199 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-08T22:23:17.321 INFO:teuthology.kill:2021-04-08 22:23:17,321.321 INFO:teuthology.orchestra.console:Power off smithi035
2021-04-08T22:23:17.344 INFO:teuthology.kill:2021-04-08 22:23:17,344.344 INFO:teuthology.orchestra.console:Power off smithi066
2021-04-08T22:23:25.721 INFO:teuthology.kill:2021-04-08 22:23:25,720.720 INFO:teuthology.orchestra.console:Power off for smithi066 completed
2021-04-08T22:23:29.826 INFO:teuthology.kill:2021-04-08 22:23:29,826.826 INFO:teuthology.orchestra.console:Power off for smithi035 completed
2021-04-08T22:23:30.056 INFO:teuthology.task.internal:roles: ubuntu@smithi035.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2']
2021-04-08T22:23:30.057 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0']
2021-04-08T22:23:30.057 INFO:teuthology.misc:Compressing logs...
2021-04-08T22:23:30.057 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-04-08T22:24:30.061 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/remote.py", line 215, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-04-08T22:24:30.115 INFO:teuthology.kill:No teuthology processes running
2021-04-08T22:24:31.244 INFO:teuthology.kill:Nuking machines: ['smithi035', 'smithi066']
2021-04-08T22:24:32.221 INFO:teuthology.kill:2021-04-08 22:24:32,220.220 INFO:teuthology.nuke:targets:
2021-04-08T22:24:32.222 INFO:teuthology.kill:  smithi035.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDK93THBCon6CuiF99H1IPX/ZEBZ23tlpzEiu/2V9aUUkxlrRwPQFWd2lvAqE4T2aXGmAE4p9ldr15ZRd7DJgEONsvjlH1rBbga7B3Klf69kcfPAY+/y9lUP2WdY6E1Yp1j87Z5dW4meLI/mRZdkkCP6nJZZ9P98sb2mZHV8Kt71G+dGFx1d+HS6Vcw8Bk7eOimqhvGYW7uE1nGK3VJPqdZ2A5oFhWcazG9fQspbVTxhvNQQEUA3WUmQn6Tw5ZXjBSsweQH4xxvAYEuQkU5m6IQ1bbOrl+xRsSl0FW7+n+ewBc7lqG0HZrYNPqjVnyIlTDTOHkmhDUSIK9u0/k3AzNh4a3iLLbszTN+WZLa5RXPh15uudpuSOfl+Tttzt6PNfZgNgeQp342RRd06uaaqPJPlS1nCFTqH+AmrqePK0nKP74ONMT9UH1Z68KuQ6N+kYOplLWWACKPRUYo5C2op8f+mSzGFH9Trp58k7PGPzGuzc0P/RTTsCdrkWfTway5GT0=
2021-04-08T22:24:32.222 INFO:teuthology.kill:  smithi066.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIpwfnwSi8MFysCVt9RGpHY8OCWJy46pghnS/f3iGpRC0tHI/MuXS1Rv4ZHxjOMkXSY1KcpgBHQ4B6R4cDFpvuxBvBmiKYpaasBWX5FkPtcAd6PPBKx0nFU0JTuU59ERRkBWtzwZUVn5u7Kp3KdjOtsfKqKjgydVTApYs70Yx6pdbj/Yt02fzyol98H6sD9EqYgA9GhBn4mq26/2M+kM67/e70WgyLFdsTS/vAM5BYVx70yPKatyYFiVJaEVF5gHclR2N7PpTaW4VsGyP3Tc40unHE/YnXv8NGSd4eX1VnN89uyeqx11gwwNfy9SLk4+vl3UoCo8CxddRrA8PepOP5LhRTkmCzZJvRRA1jYjegjcVVI8fZGPXRHYQSGsk26L+Z9VI1nKNarXTQ3JVOvvSuC6ZmAU704TkOXgiCGLXoTnX74VqY7zfhY+BEC7hxXfQNphGXjDR1xqqZRcbZDiYq4e+egorWCITs/L8tJy5kvaQyOmeOB/DWD3UzIqtnuEc=
2021-04-08T22:24:32.242 INFO:teuthology.kill:2021-04-08 22:24:32,242.242 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-08T22:24:32.255 INFO:teuthology.kill:2021-04-08 22:24:32,255.255 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-08T22:24:32.334 INFO:teuthology.kill:2021-04-08 22:24:32,334.334 INFO:teuthology.orchestra.console:Power off smithi035
2021-04-08T22:24:32.347 INFO:teuthology.kill:2021-04-08 22:24:32,347.347 INFO:teuthology.orchestra.console:Power off smithi066
2021-04-08T22:24:32.477 INFO:teuthology.kill:2021-04-08 22:24:32,477.477 INFO:teuthology.orchestra.console:Power off for smithi035 completed
2021-04-08T22:24:32.488 INFO:teuthology.kill:2021-04-08 22:24:32,487.487 INFO:teuthology.orchestra.console:Power off for smithi066 completed
2021-04-08T22:24:32.629 INFO:teuthology.kill:2021-04-08 22:24:32,628.628 INFO:teuthology.lock.ops:unlocked smithi035.front.sepia.ceph.com
2021-04-08T22:24:32.645 INFO:teuthology.kill:2021-04-08 22:24:32,645.645 INFO:teuthology.lock.ops:unlocked smithi066.front.sepia.ceph.com
2021-04-08T22:26:32.940 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-04-08T22:26:33.030 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-04-08T22:26:33.030 INFO:teuthology.nuke:Checking targets against current locks
2021-04-08T22:26:33.129 INFO:teuthology.nuke:Not nuking smithi066.front.sepia.ceph.com because description doesn't match
2021-04-08T22:26:33.129 INFO:teuthology.nuke:Not nuking smithi035.front.sepia.ceph.com because description doesn't match