2021-03-13T10:15:22.857 INFO:root:teuthology version: 1.1.0-b9656917 2021-03-13T10:15:22.966 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging 2021-03-13T10:15:22.967 INFO:teuthology.lock.ops:Updating [ubuntu@smithi110.front.sepia.ceph.com]: reset os type and version on server 2021-03-13T10:15:22.968 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2021-03-13T10:15:22.983 INFO:teuthology.lock.ops:Node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging is complete 2021-03-13T10:15:22.984 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi190.front.sepia.ceph.com' reimaging 2021-03-13T10:15:22.985 INFO:teuthology.lock.ops:Updating [ubuntu@smithi190.front.sepia.ceph.com]: reset os type and version on server 2021-03-13T10:15:22.985 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2021-03-13T10:15:23.000 INFO:teuthology.lock.ops:Node 'ubuntu@smithi190.front.sepia.ceph.com' reimaging is complete 2021-03-13T10:15:23.213 INFO:teuthology.provision.fog.smithi110:Scheduling deploy of centos 8.3 2021-03-13T10:15:23.285 INFO:teuthology.provision.fog.smithi190:Scheduling deploy of centos 8.3 2021-03-13T10:15:23.652 INFO:teuthology.orchestra.console:Power off smithi110 2021-03-13T10:15:23.692 INFO:teuthology.orchestra.console:Power off smithi190 2021-03-13T10:15:23.832 INFO:teuthology.orchestra.console:Power off for smithi190 completed 2021-03-13T10:15:23.933 INFO:teuthology.orchestra.console:Power on smithi190 2021-03-13T10:15:32.023 INFO:teuthology.orchestra.console:Power off for smithi110 completed 2021-03-13T10:15:32.124 INFO:teuthology.orchestra.console:Power on smithi110 2021-03-13T10:15:32.298 INFO:teuthology.orchestra.console:Power on for smithi190 completed 2021-03-13T10:15:32.400 INFO:teuthology.provision.fog.smithi190:Waiting for deploy to finish 2021-03-13T10:15:40.496 INFO:teuthology.orchestra.console:Power on for smithi110 completed 2021-03-13T10:15:40.598 INFO:teuthology.provision.fog.smithi110:Waiting for deploy to finish 2021-03-13T10:19:47.024 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-13T10:20:07.082 INFO:teuthology.provision.fog.smithi110:Node is ready 2021-03-13T10:20:07.100 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110.front.sepia.ceph.com 2021-03-13T10:20:07.158 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110 smithi110.front.sepia.ceph.com smithi110 2021-03-13T10:20:07.503 INFO:teuthology.provision.fog.smithi110:Deploy complete! 2021-03-13T10:20:13.366 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-13T10:20:13.669 INFO:teuthology.provision.fog.smithi190:Node is ready 2021-03-13T10:20:13.726 INFO:teuthology.orchestra.run.smithi190.stdout:smithi190.front.sepia.ceph.com 2021-03-13T10:20:13.786 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.15.190 smithi190.front.sepia.ceph.com smithi190 2021-03-13T10:20:14.126 INFO:teuthology.provision.fog.smithi190:Deploy complete! 2021-03-13T10:20:14.392 INFO:teuthology.lock.ops:Checking smithi110.front.sepia.ceph.com 2021-03-13T10:20:14.398 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-13T10:20:14.422 INFO:teuthology.lock.ops:Checking smithi190.front.sepia.ceph.com 2021-03-13T10:20:14.422 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-13T10:20:14.438 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: set os type and version on server 2021-03-13T10:20:14.439 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-13T10:20:14.736 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2021-03-13T10:20:14.790 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Linux" 2021-03-13T10:20:14.790 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="8" 2021-03-13T10:20:14.790 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2021-03-13T10:20:14.791 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2021-03-13T10:20:14.791 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="8" 2021-03-13T10:20:14.791 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el8" 2021-03-13T10:20:14.791 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Linux 8" 2021-03-13T10:20:14.792 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2021-03-13T10:20:14.792 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-03-13T10:20:14.792 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2021-03-13T10:20:14.792 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-03-13T10:20:14.793 INFO:teuthology.orchestra.run.smithi110.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-03-13T10:20:14.793 INFO:teuthology.orchestra.run.smithi110.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-03-13T10:20:14.794 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2021-03-13T10:20:14.811 INFO:teuthology.lock.ops:Updating [smithi190.front.sepia.ceph.com]: set os type and version on server 2021-03-13T10:20:14.812 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-13T10:20:15.087 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2021-03-13T10:20:15.140 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Linux" 2021-03-13T10:20:15.140 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="8" 2021-03-13T10:20:15.141 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2021-03-13T10:20:15.141 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2021-03-13T10:20:15.141 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="8" 2021-03-13T10:20:15.141 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el8" 2021-03-13T10:20:15.142 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Linux 8" 2021-03-13T10:20:15.142 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2021-03-13T10:20:15.142 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-03-13T10:20:15.142 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2021-03-13T10:20:15.143 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-03-13T10:20:15.143 INFO:teuthology.orchestra.run.smithi190.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-03-13T10:20:15.143 INFO:teuthology.orchestra.run.smithi190.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-03-13T10:20:15.144 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2021-03-13T10:20:15.231 INFO:teuthology.dispatcher.supervisor:Running job 5961092 2021-03-13T10:20:15.231 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/bin/teuthology -v --owner scheduled_kchai@teuthology --archive /home/teuthworker/archive/kchai-2021-03-13_08:16:32-rados-wip-kefu-testing-2021-03-13-1449-distro-basic-smithi/5961092 --name kchai-2021-03-13_08:16:32-rados-wip-kefu-testing-2021-03-13-1449-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-agent-big} -- /home/teuthworker/archive/kchai-2021-03-13_08:16:32-rados-wip-kefu-testing-2021-03-13-1449-distro-basic-smithi/5961092/orig.config.yaml 2021-03-13T10:20:15.235 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/kchai-2021-03-13_08:16:32-rados-wip-kefu-testing-2021-03-13-1449-distro-basic-smithi/5961092 2021-03-13T10:20:15.236 INFO:teuthology.dispatcher.supervisor:Job PID: 14044 2021-03-13T10:20:15.236 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-03-13T22:21:07.027 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-03-13T22:21:07.836 INFO:teuthology.kill:Killing Pids: {14044} 2021-03-13T22:21:09.879 INFO:teuthology.kill:Nuking machines: ['smithi110', 'smithi190'] 2021-03-13T22:21:11.534 INFO:teuthology.kill:2021-03-13 22:21:11,534.534 INFO:teuthology.nuke:targets: 2021-03-13T22:21:11.535 INFO:teuthology.kill: smithi110.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMyk8FXO+3TqWqZgpO4c7gsliIdltnlpnm7djTTrpZjotrKGFzlY5rxehjoVqBk8k26GMMXYB0t+SztnE0A7MInbmOlTVTbS3BFYr4C+fzs26vLcIKstwhqfOB4WGst5tsI+S7hnTCYiHvorhIioYTwRtYGX+kX9bntVACbWt9ajsU9chJQdhpUxt9z8N7P+9upCS3lUeAjYu+IPWy04Bahw/ZxrJQWyWK2jt0LHTbBtSB2ESmvla09PXPxYKo7dVYYWqHeGBpPE/vQdOgPy7i7mk+j3F0RwDpAOjAaQxRVXsFh5YsAk74y5YRuqSjb67MYd+zIhDg9CBY7Yn+WqfDtrJvbqnAUBSXGYC+NMNAmycs4kElajUpXhnBEcEPGJnWJdS/zBmn7EEp+fE9gjmQQNUqkJn//E0bKIPnyS+O1h5+76owyk1M4Y7csd1HX08zA8IE1rHyzA6pCYttsDGTPPJo1m5HRrrrRbUxcm+w0kdAmaxlSED3Cgg9SBF/bu8= 2021-03-13T22:21:11.535 INFO:teuthology.kill: smithi190.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3toDW6vmcVcAZonW50mzPRdYGKUZKeqXp/Dbl8iO+M/lg8TNNgq5RxsypfEcps2rlb/p4Zidk8r5pj/A0Qn/15VyZ2uKjfAzS8Y8VpGa0/apRirHuQWb0Fe8O4UBf8XgclvAFddYBsOBOrb7d4cMMXNQ3RZc9OlKNhuBOzatEzg/1rtUp2+bVmMwRw7WYTgN5JEDO+qNmWBcmiqnsSOFiDYDofjVEAXGBFQFV2XE0kIn3YAS2Y34fewGEExF0Y8hqlkpuWfX4Yz6Hep8o7x5Ludk9PWx8oFtI9S4f4LaRnHxLJ0yz+AHz6z/AGLGHv33uqbk59y4CCQ/t1CESa6/VBJk5wh+4B0Fv7jnU9oBeTAdcfawCU+c5GC4JrBpw1tLXhAwbIeQVeWvSmIl9RrpXy8cT/O2+0BKR4SqUKfHs1tYLHwwMT+G6StBHalwinOHkp4nvgRN2hGyB6mt81diZwD5O8j17hXEfRPQ+Z2wOAz14mbkkFSfDfv3onkxdkJM= 2021-03-13T22:21:11.536 INFO:teuthology.kill:2021-03-13 22:21:11,534.534 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-13T22:21:11.537 INFO:teuthology.kill:2021-03-13 22:21:11,536.536 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-13T22:21:11.650 INFO:teuthology.kill:2021-03-13 22:21:11,650.650 INFO:teuthology.orchestra.console:Power off smithi110 2021-03-13T22:21:11.708 INFO:teuthology.kill:2021-03-13 22:21:11,707.707 INFO:teuthology.orchestra.console:Power off smithi190 2021-03-13T22:21:20.160 INFO:teuthology.kill:2021-03-13 22:21:20,160.160 INFO:teuthology.orchestra.console:Power off for smithi110 completed 2021-03-13T22:21:24.315 INFO:teuthology.kill:2021-03-13 22:21:24,314.314 INFO:teuthology.orchestra.console:Power off for smithi190 completed 2021-03-13T22:21:24.631 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-03-13T22:21:24.632 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-03-13T22:21:24.633 INFO:teuthology.misc:Compressing logs... 2021-03-13T22:21:24.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-13T22:22:24.699 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/misc.py", line 1374, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-03-13T22:22:24.776 INFO:teuthology.kill:No teuthology processes running 2021-03-13T22:22:26.384 INFO:teuthology.kill:Nuking machines: ['smithi110', 'smithi190'] 2021-03-13T22:22:28.094 INFO:teuthology.kill:2021-03-13 22:22:28,094.094 INFO:teuthology.nuke:targets: 2021-03-13T22:22:28.095 INFO:teuthology.kill: smithi110.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMyk8FXO+3TqWqZgpO4c7gsliIdltnlpnm7djTTrpZjotrKGFzlY5rxehjoVqBk8k26GMMXYB0t+SztnE0A7MInbmOlTVTbS3BFYr4C+fzs26vLcIKstwhqfOB4WGst5tsI+S7hnTCYiHvorhIioYTwRtYGX+kX9bntVACbWt9ajsU9chJQdhpUxt9z8N7P+9upCS3lUeAjYu+IPWy04Bahw/ZxrJQWyWK2jt0LHTbBtSB2ESmvla09PXPxYKo7dVYYWqHeGBpPE/vQdOgPy7i7mk+j3F0RwDpAOjAaQxRVXsFh5YsAk74y5YRuqSjb67MYd+zIhDg9CBY7Yn+WqfDtrJvbqnAUBSXGYC+NMNAmycs4kElajUpXhnBEcEPGJnWJdS/zBmn7EEp+fE9gjmQQNUqkJn//E0bKIPnyS+O1h5+76owyk1M4Y7csd1HX08zA8IE1rHyzA6pCYttsDGTPPJo1m5HRrrrRbUxcm+w0kdAmaxlSED3Cgg9SBF/bu8= 2021-03-13T22:22:28.095 INFO:teuthology.kill: smithi190.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3toDW6vmcVcAZonW50mzPRdYGKUZKeqXp/Dbl8iO+M/lg8TNNgq5RxsypfEcps2rlb/p4Zidk8r5pj/A0Qn/15VyZ2uKjfAzS8Y8VpGa0/apRirHuQWb0Fe8O4UBf8XgclvAFddYBsOBOrb7d4cMMXNQ3RZc9OlKNhuBOzatEzg/1rtUp2+bVmMwRw7WYTgN5JEDO+qNmWBcmiqnsSOFiDYDofjVEAXGBFQFV2XE0kIn3YAS2Y34fewGEExF0Y8hqlkpuWfX4Yz6Hep8o7x5Ludk9PWx8oFtI9S4f4LaRnHxLJ0yz+AHz6z/AGLGHv33uqbk59y4CCQ/t1CESa6/VBJk5wh+4B0Fv7jnU9oBeTAdcfawCU+c5GC4JrBpw1tLXhAwbIeQVeWvSmIl9RrpXy8cT/O2+0BKR4SqUKfHs1tYLHwwMT+G6StBHalwinOHkp4nvgRN2hGyB6mt81diZwD5O8j17hXEfRPQ+Z2wOAz14mbkkFSfDfv3onkxdkJM= 2021-03-13T22:22:28.142 INFO:teuthology.kill:2021-03-13 22:22:28,140.140 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-13T22:22:28.142 INFO:teuthology.kill:2021-03-13 22:22:28,142.142 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-13T22:22:28.234 INFO:teuthology.kill:2021-03-13 22:22:28,233.233 INFO:teuthology.orchestra.console:Power off smithi110 2021-03-13T22:22:28.287 INFO:teuthology.kill:2021-03-13 22:22:28,287.287 INFO:teuthology.orchestra.console:Power off smithi190 2021-03-13T22:22:28.495 INFO:teuthology.kill:2021-03-13 22:22:28,495.495 INFO:teuthology.orchestra.console:Power off for smithi110 completed 2021-03-13T22:22:28.539 INFO:teuthology.kill:2021-03-13 22:22:28,539.539 INFO:teuthology.orchestra.console:Power off for smithi190 completed 2021-03-13T22:22:28.637 INFO:teuthology.kill:2021-03-13 22:22:28,636.636 INFO:teuthology.lock.ops:unlocked smithi110.front.sepia.ceph.com 2021-03-13T22:22:28.678 INFO:teuthology.kill:2021-03-13 22:22:28,678.678 INFO:teuthology.lock.ops:unlocked smithi190.front.sepia.ceph.com 2021-03-13T22:24:29.093 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15