2021-04-14T07:12:01.067 INFO:root:teuthology version: 1.1.0-2713a3cd
2021-04-14T07:12:01.185 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi077.front.sepia.ceph.com' reimaging
2021-04-14T07:12:01.187 INFO:teuthology.lock.ops:Updating [ubuntu@smithi077.front.sepia.ceph.com]: reset os type and version on server
2021-04-14T07:12:01.187 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server
2021-04-14T07:12:01.207 INFO:teuthology.lock.ops:Node 'ubuntu@smithi077.front.sepia.ceph.com' reimaging is complete
2021-04-14T07:12:01.207 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi198.front.sepia.ceph.com' reimaging
2021-04-14T07:12:01.208 INFO:teuthology.lock.ops:Updating [ubuntu@smithi198.front.sepia.ceph.com]: reset os type and version on server
2021-04-14T07:12:01.208 INFO:teuthology.lock.ops:Updating smithi198.front.sepia.ceph.com on lock server
2021-04-14T07:12:01.225 INFO:teuthology.lock.ops:Node 'ubuntu@smithi198.front.sepia.ceph.com' reimaging is complete
2021-04-14T07:12:01.409 INFO:teuthology.provision.fog.smithi077:Scheduling deploy of centos 8.2
2021-04-14T07:12:01.423 INFO:teuthology.provision.fog.smithi198:Scheduling deploy of centos 8.2
2021-04-14T07:12:01.839 INFO:teuthology.orchestra.console:Power off smithi077
2021-04-14T07:12:01.928 INFO:teuthology.orchestra.console:Power off smithi198
2021-04-14T07:12:10.357 INFO:teuthology.orchestra.console:Power off for smithi077 completed
2021-04-14T07:12:10.460 INFO:teuthology.orchestra.console:Power on smithi077
2021-04-14T07:12:14.522 INFO:teuthology.orchestra.console:Power off for smithi198 completed
2021-04-14T07:12:14.624 INFO:teuthology.orchestra.console:Power on smithi198
2021-04-14T07:12:14.756 INFO:teuthology.orchestra.console:Power on for smithi077 completed
2021-04-14T07:12:14.857 INFO:teuthology.provision.fog.smithi077:Waiting for deploy to finish
2021-04-14T07:12:18.961 INFO:teuthology.orchestra.console:Power on for smithi198 completed
2021-04-14T07:12:19.063 INFO:teuthology.provision.fog.smithi198:Waiting for deploy to finish
2021-04-14T07:16:51.000 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-04-14T07:16:51.281 INFO:teuthology.provision.fog.smithi198:Node is ready
2021-04-14T07:16:51.296 INFO:teuthology.orchestra.run.smithi198.stdout:smithi198.front.sepia.ceph.com
2021-04-14T07:16:51.350 INFO:teuthology.orchestra.run.smithi198.stdout:172.21.15.198 smithi198.front.sepia.ceph.com smithi198
2021-04-14T07:16:51.664 INFO:teuthology.provision.fog.smithi198:Deploy complete!
2021-04-14T07:16:55.859 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-04-14T07:16:56.161 INFO:teuthology.provision.fog.smithi077:Node is ready
2021-04-14T07:16:56.176 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077.front.sepia.ceph.com
2021-04-14T07:16:56.231 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77 smithi077.front.sepia.ceph.com smithi077
2021-04-14T07:16:56.553 INFO:teuthology.provision.fog.smithi077:Deploy complete!
2021-04-14T07:16:56.703 INFO:teuthology.lock.ops:Checking smithi077.front.sepia.ceph.com
2021-04-14T07:16:56.704 INFO:teuthology.lock.ops:New key found. Updating...
2021-04-14T07:16:56.720 INFO:teuthology.lock.ops:Checking smithi198.front.sepia.ceph.com
2021-04-14T07:16:56.720 INFO:teuthology.lock.ops:New key found. Updating...
2021-04-14T07:16:56.735 INFO:teuthology.lock.ops:Updating [smithi077.front.sepia.ceph.com]: set os type and version on server
2021-04-14T07:16:56.735 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-04-14T07:16:57.039 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64
2021-04-14T07:16:57.093 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Linux"
2021-04-14T07:16:57.093 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="8 (Core)"
2021-04-14T07:16:57.093 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos"
2021-04-14T07:16:57.093 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora"
2021-04-14T07:16:57.094 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="8"
2021-04-14T07:16:57.094 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el8"
2021-04-14T07:16:57.094 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Linux 8 (Core)"
2021-04-14T07:16:57.095 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31"
2021-04-14T07:16:57.095 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-04-14T07:16:57.095 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://www.centos.org/"
2021-04-14T07:16:57.095 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-04-14T07:16:57.095 INFO:teuthology.orchestra.run.smithi077.stdout:
2021-04-14T07:16:57.096 INFO:teuthology.orchestra.run.smithi077.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-04-14T07:16:57.096 INFO:teuthology.orchestra.run.smithi077.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-04-14T07:16:57.096 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="centos"
2021-04-14T07:16:57.097 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8"
2021-04-14T07:16:57.097 INFO:teuthology.orchestra.run.smithi077.stdout:
2021-04-14T07:16:57.098 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server
2021-04-14T07:16:57.116 INFO:teuthology.lock.ops:Updating [smithi198.front.sepia.ceph.com]: set os type and version on server
2021-04-14T07:16:57.117 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-04-14T07:16:57.396 INFO:teuthology.orchestra.run.smithi198.stdout:x86_64
2021-04-14T07:16:57.450 INFO:teuthology.orchestra.run.smithi198.stdout:NAME="CentOS Linux"
2021-04-14T07:16:57.451 INFO:teuthology.orchestra.run.smithi198.stdout:VERSION="8 (Core)"
2021-04-14T07:16:57.451 INFO:teuthology.orchestra.run.smithi198.stdout:ID="centos"
2021-04-14T07:16:57.451 INFO:teuthology.orchestra.run.smithi198.stdout:ID_LIKE="rhel fedora"
2021-04-14T07:16:57.452 INFO:teuthology.orchestra.run.smithi198.stdout:VERSION_ID="8"
2021-04-14T07:16:57.452 INFO:teuthology.orchestra.run.smithi198.stdout:PLATFORM_ID="platform:el8"
2021-04-14T07:16:57.452 INFO:teuthology.orchestra.run.smithi198.stdout:PRETTY_NAME="CentOS Linux 8 (Core)"
2021-04-14T07:16:57.452 INFO:teuthology.orchestra.run.smithi198.stdout:ANSI_COLOR="0;31"
2021-04-14T07:16:57.452 INFO:teuthology.orchestra.run.smithi198.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-04-14T07:16:57.453 INFO:teuthology.orchestra.run.smithi198.stdout:HOME_URL="https://www.centos.org/"
2021-04-14T07:16:57.453 INFO:teuthology.orchestra.run.smithi198.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-04-14T07:16:57.453 INFO:teuthology.orchestra.run.smithi198.stdout:
2021-04-14T07:16:57.453 INFO:teuthology.orchestra.run.smithi198.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-04-14T07:16:57.453 INFO:teuthology.orchestra.run.smithi198.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-04-14T07:16:57.454 INFO:teuthology.orchestra.run.smithi198.stdout:REDHAT_SUPPORT_PRODUCT="centos"
2021-04-14T07:16:57.454 INFO:teuthology.orchestra.run.smithi198.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8"
2021-04-14T07:16:57.454 INFO:teuthology.orchestra.run.smithi198.stdout:
2021-04-14T07:16:57.455 INFO:teuthology.lock.ops:Updating smithi198.front.sepia.ceph.com on lock server
2021-04-14T07:16:57.549 INFO:teuthology.dispatcher.supervisor:Running job 6044948
2021-04-14T07:16:57.550 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/virtualenv/bin/teuthology -v --owner scheduled_gregf@teuthology --archive /home/teuthworker/archive/gregf-2021-04-14_03:25:22-upgrade:octopus-x-wip-mon-quorum-leader-413-distro-basic-smithi/6044948 --name gregf-2021-04-14_03:25:22-upgrade:octopus-x-wip-mon-quorum-leader-413-distro-basic-smithi --description upgrade:octopus-x/stress-split/{0-distro/centos_8.2_kubic_stable 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} -- /home/teuthworker/archive/gregf-2021-04-14_03:25:22-upgrade:octopus-x-wip-mon-quorum-leader-413-distro-basic-smithi/6044948/orig.config.yaml
2021-04-14T07:16:57.555 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/gregf-2021-04-14_03:25:22-upgrade:octopus-x-wip-mon-quorum-leader-413-distro-basic-smithi/6044948
2021-04-14T07:16:57.556 INFO:teuthology.dispatcher.supervisor:Job PID: 20651
2021-04-14T07:16:57.557 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-04-14T19:19:06.168 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-04-14T19:19:06.373 INFO:teuthology.kill:Killing Pids: {20651}
2021-04-14T19:19:07.384 INFO:teuthology.kill:Nuking machines: ['smithi077', 'smithi198']
2021-04-14T19:19:08.310 INFO:teuthology.kill:2021-04-14 19:19:08,309.309 INFO:teuthology.nuke:targets:
2021-04-14T19:19:08.310 INFO:teuthology.kill:  smithi077.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCa2S+btTQAW2M0+T5+FhCX6L22VNmNccGgIPdmJ8hy3qZhzv56s7f9zWsSJye+iA9SHrg1tjFBturZxJM7tjavqm6Ty7dpJhWT/Uy86xavZt2OTelbAG1eJU23MC8tWxhBQpupi2mi0BMM+gMCkBjql4ch5pRHKX8STfDIeyK/lMLwvbv0VjYXTYdkbCRAzdwCQc3DCjZGdeW5ZyeMtQmvK7Ol8GSw06G+1BHQ3iU0DLWAL7WSWN37hgtdxynIpUSgydOtSSdQZJF+EmcrLjuJSksC1nVBxDa4ClEivi6dXyR3744o2DzKxXCD/4YcAyjsEoE/0YcXh42EX4HvYhfd1/e5NzTHSFliE68b7dJCcGhNBCjUk+RcLWGzJ0aQ84tXHB1cO3GBVR3EeD+RYUqzEBYAx3AjEKMr5uRJhjArWsG4x4IJlkxSU3kuEoMzZQLtCQkrcYSEXQYybQIY2w4BessNigMhaDtu/pADWqJHwMlY2yU/xG+CRkSImJTVcis=
2021-04-14T19:19:08.311 INFO:teuthology.kill:  smithi198.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMgGkyDvfoR46P2kbBOy8LV20srCP0vMwavdbWkqJrCtAeOcC3SCgEJSdsdnfbSmS5YpSu+YouV+XzLIDR6H46gaLjGjgzuCmHwzMVRrgugRvyQniK53Fgc+PcxeycWxNidO0Zlollrm4gMdX19JTY3i8JNnqlun18YXl4MN4EALHsevbqfwJ92prjOO/OdRORpRW5MPNZ9/H0/6/jW/rJcVJYkVSeFSO5ASKRAAsTvVWF9kfWooySWnKcl4eUJdNOLFYzOononsaGV2x62cFzNIcgkqtm3YcM5FXuUEqVT0wlJTvWJm12PBeuUA6tGf0iMUV6scFH9YGIH1tbHrCbxx7icuePIpcMdy216bzHGejNoEQ+TKovKi+ecXCJEr60csP6noLzja2oEbhvoEUm4aGIzIGvVYXDQoBnXzSRO49LHkgN/hRJjciA1mUwCzxkHyOshyk2UeEebmV7ag/GpD+jj5E7BTX02NlspjoqJa8pkaU1QoxL0FmVeNginB8=
2021-04-14T19:19:08.311 INFO:teuthology.kill:2021-04-14 19:19:08,310.310 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-14T19:19:08.312 INFO:teuthology.kill:2021-04-14 19:19:08,312.312 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-14T19:19:08.380 INFO:teuthology.kill:2021-04-14 19:19:08,380.380 INFO:teuthology.orchestra.console:Power off smithi077
2021-04-14T19:19:08.394 INFO:teuthology.kill:2021-04-14 19:19:08,394.394 INFO:teuthology.orchestra.console:Power off smithi198
2021-04-14T19:19:16.756 INFO:teuthology.kill:2021-04-14 19:19:16,755.755 INFO:teuthology.orchestra.console:Power off for smithi198 completed
2021-04-14T19:19:20.859 INFO:teuthology.kill:2021-04-14 19:19:20,859.859 INFO:teuthology.orchestra.console:Power off for smithi077 completed
2021-04-14T19:19:21.084 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a']
2021-04-14T19:19:21.085 INFO:teuthology.task.internal:roles: ubuntu@smithi198.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b']
2021-04-14T19:19:21.085 INFO:teuthology.misc:Compressing logs...
2021-04-14T19:19:21.086 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-04-14T19:19:39.611 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/remote.py", line 215, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-04-14T19:19:39.701 INFO:teuthology.kill:No teuthology processes running
2021-04-14T19:19:40.651 INFO:teuthology.kill:Nuking machines: ['smithi077', 'smithi198']
2021-04-14T19:19:41.559 INFO:teuthology.kill:2021-04-14 19:19:41,558.558 INFO:teuthology.nuke:targets:
2021-04-14T19:19:41.560 INFO:teuthology.kill:  smithi077.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCa2S+btTQAW2M0+T5+FhCX6L22VNmNccGgIPdmJ8hy3qZhzv56s7f9zWsSJye+iA9SHrg1tjFBturZxJM7tjavqm6Ty7dpJhWT/Uy86xavZt2OTelbAG1eJU23MC8tWxhBQpupi2mi0BMM+gMCkBjql4ch5pRHKX8STfDIeyK/lMLwvbv0VjYXTYdkbCRAzdwCQc3DCjZGdeW5ZyeMtQmvK7Ol8GSw06G+1BHQ3iU0DLWAL7WSWN37hgtdxynIpUSgydOtSSdQZJF+EmcrLjuJSksC1nVBxDa4ClEivi6dXyR3744o2DzKxXCD/4YcAyjsEoE/0YcXh42EX4HvYhfd1/e5NzTHSFliE68b7dJCcGhNBCjUk+RcLWGzJ0aQ84tXHB1cO3GBVR3EeD+RYUqzEBYAx3AjEKMr5uRJhjArWsG4x4IJlkxSU3kuEoMzZQLtCQkrcYSEXQYybQIY2w4BessNigMhaDtu/pADWqJHwMlY2yU/xG+CRkSImJTVcis=
2021-04-14T19:19:41.560 INFO:teuthology.kill:  smithi198.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMgGkyDvfoR46P2kbBOy8LV20srCP0vMwavdbWkqJrCtAeOcC3SCgEJSdsdnfbSmS5YpSu+YouV+XzLIDR6H46gaLjGjgzuCmHwzMVRrgugRvyQniK53Fgc+PcxeycWxNidO0Zlollrm4gMdX19JTY3i8JNnqlun18YXl4MN4EALHsevbqfwJ92prjOO/OdRORpRW5MPNZ9/H0/6/jW/rJcVJYkVSeFSO5ASKRAAsTvVWF9kfWooySWnKcl4eUJdNOLFYzOononsaGV2x62cFzNIcgkqtm3YcM5FXuUEqVT0wlJTvWJm12PBeuUA6tGf0iMUV6scFH9YGIH1tbHrCbxx7icuePIpcMdy216bzHGejNoEQ+TKovKi+ecXCJEr60csP6noLzja2oEbhvoEUm4aGIzIGvVYXDQoBnXzSRO49LHkgN/hRJjciA1mUwCzxkHyOshyk2UeEebmV7ag/GpD+jj5E7BTX02NlspjoqJa8pkaU1QoxL0FmVeNginB8=
2021-04-14T19:19:41.580 INFO:teuthology.kill:2021-04-14 19:19:41,580.580 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-14T19:19:41.592 INFO:teuthology.kill:2021-04-14 19:19:41,592.592 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-14T19:19:41.655 INFO:teuthology.kill:2021-04-14 19:19:41,655.655 INFO:teuthology.orchestra.console:Power off smithi077
2021-04-14T19:19:41.668 INFO:teuthology.kill:2021-04-14 19:19:41,667.667 INFO:teuthology.orchestra.console:Power off smithi198
2021-04-14T19:19:41.792 INFO:teuthology.kill:2021-04-14 19:19:41,792.792 INFO:teuthology.orchestra.console:Power off for smithi077 completed
2021-04-14T19:19:41.804 INFO:teuthology.kill:2021-04-14 19:19:41,803.803 INFO:teuthology.orchestra.console:Power off for smithi198 completed
2021-04-14T19:19:41.938 INFO:teuthology.kill:2021-04-14 19:19:41,937.937 INFO:teuthology.lock.ops:unlocked smithi077.front.sepia.ceph.com
2021-04-14T19:19:41.957 INFO:teuthology.kill:2021-04-14 19:19:41,956.956 INFO:teuthology.lock.ops:unlocked smithi198.front.sepia.ceph.com
2021-04-14T19:22:17.134 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15