2021-08-11T15:01:28.626 INFO:root:teuthology version: 1.1.0-321319b1 2021-08-11T15:01:28.727 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi012.front.sepia.ceph.com' reimaging 2021-08-11T15:01:28.728 INFO:teuthology.lock.ops:Updating [ubuntu@smithi012.front.sepia.ceph.com]: reset os type and version on server 2021-08-11T15:01:28.729 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2021-08-11T15:01:28.745 INFO:teuthology.lock.ops:Node 'ubuntu@smithi012.front.sepia.ceph.com' reimaging is complete 2021-08-11T15:01:28.745 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging 2021-08-11T15:01:28.746 INFO:teuthology.lock.ops:Updating [ubuntu@smithi136.front.sepia.ceph.com]: reset os type and version on server 2021-08-11T15:01:28.746 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2021-08-11T15:01:28.763 INFO:teuthology.lock.ops:Node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging is complete 2021-08-11T15:01:28.964 INFO:teuthology.provision.fog.smithi012:Scheduling deploy of centos 8.2 2021-08-11T15:01:28.975 INFO:teuthology.provision.fog.smithi136:Scheduling deploy of centos 8.2 2021-08-11T15:01:29.310 INFO:teuthology.orchestra.console:Power off smithi012 2021-08-11T15:01:29.324 INFO:teuthology.orchestra.console:Power off smithi136 2021-08-11T15:01:29.453 INFO:teuthology.orchestra.console:Power off for smithi012 completed 2021-08-11T15:01:29.462 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2021-08-11T15:01:29.555 INFO:teuthology.orchestra.console:Power on smithi012 2021-08-11T15:01:29.563 INFO:teuthology.orchestra.console:Power on smithi136 2021-08-11T15:01:37.918 INFO:teuthology.orchestra.console:Power on for smithi012 completed 2021-08-11T15:01:37.922 INFO:teuthology.orchestra.console:Power on for smithi136 completed 2021-08-11T15:01:38.020 INFO:teuthology.provision.fog.smithi012:Waiting for deploy to finish 2021-08-11T15:01:38.024 INFO:teuthology.provision.fog.smithi136:Waiting for deploy to finish 2021-08-11T15:06:10.085 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-08-11T15:06:10.374 INFO:teuthology.provision.fog.smithi136:Node is ready 2021-08-11T15:06:10.389 INFO:teuthology.orchestra.run.smithi136.stdout:smithi136.front.sepia.ceph.com 2021-08-11T15:06:10.444 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.15.136 smithi136.front.sepia.ceph.com smithi136 2021-08-11T15:06:10.761 INFO:teuthology.provision.fog.smithi136:Deploy complete! 2021-08-11T15:07:01.385 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-08-11T15:07:01.688 INFO:teuthology.provision.fog.smithi012:Node is ready 2021-08-11T15:07:01.703 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012.front.sepia.ceph.com 2021-08-11T15:07:01.760 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12 smithi012.front.sepia.ceph.com smithi012 2021-08-11T15:07:02.099 INFO:teuthology.provision.fog.smithi012:Deploy complete! 2021-08-11T15:07:02.255 INFO:teuthology.lock.ops:Checking smithi012.front.sepia.ceph.com 2021-08-11T15:07:02.256 INFO:teuthology.lock.ops:New key found. Updating... 2021-08-11T15:07:02.275 INFO:teuthology.lock.ops:Checking smithi136.front.sepia.ceph.com 2021-08-11T15:07:02.275 INFO:teuthology.lock.ops:New key found. Updating... 2021-08-11T15:07:02.300 INFO:teuthology.lock.ops:Updating [smithi012.front.sepia.ceph.com]: set os type and version on server 2021-08-11T15:07:02.300 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-11T15:07:02.568 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2021-08-11T15:07:02.622 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="CentOS Linux" 2021-08-11T15:07:02.622 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="8 (Core)" 2021-08-11T15:07:02.622 INFO:teuthology.orchestra.run.smithi012.stdout:ID="centos" 2021-08-11T15:07:02.623 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="rhel fedora" 2021-08-11T15:07:02.623 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="8" 2021-08-11T15:07:02.623 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el8" 2021-08-11T15:07:02.623 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-08-11T15:07:02.624 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2021-08-11T15:07:02.624 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-08-11T15:07:02.624 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.centos.org/" 2021-08-11T15:07:02.624 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-08-11T15:07:02.625 INFO:teuthology.orchestra.run.smithi012.stdout: 2021-08-11T15:07:02.625 INFO:teuthology.orchestra.run.smithi012.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-08-11T15:07:02.625 INFO:teuthology.orchestra.run.smithi012.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-08-11T15:07:02.625 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-08-11T15:07:02.625 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-08-11T15:07:02.626 INFO:teuthology.orchestra.run.smithi012.stdout: 2021-08-11T15:07:02.627 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2021-08-11T15:07:02.645 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: set os type and version on server 2021-08-11T15:07:02.646 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-11T15:07:02.919 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2021-08-11T15:07:02.973 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Linux" 2021-08-11T15:07:02.973 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="8 (Core)" 2021-08-11T15:07:02.973 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2021-08-11T15:07:02.974 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2021-08-11T15:07:02.974 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="8" 2021-08-11T15:07:02.974 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el8" 2021-08-11T15:07:02.974 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-08-11T15:07:02.974 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2021-08-11T15:07:02.975 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-08-11T15:07:02.975 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.centos.org/" 2021-08-11T15:07:02.975 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-08-11T15:07:02.975 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-08-11T15:07:02.976 INFO:teuthology.orchestra.run.smithi136.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-08-11T15:07:02.976 INFO:teuthology.orchestra.run.smithi136.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-08-11T15:07:02.976 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-08-11T15:07:02.976 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-08-11T15:07:02.976 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-08-11T15:07:02.977 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2021-08-11T15:07:03.052 INFO:teuthology.dispatcher.supervisor:Running job 6334693 2021-08-11T15:07:03.052 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-08-11_14:15:52-rados-wip-yuri3-testing-2021-08-09-1006-distro-basic-smithi/6334693 --name yuriw-2021-08-11_14:15:52-rados-wip-yuri3-testing-2021-08-09-1006-distro-basic-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_kubic_stable 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2021-08-11_14:15:52-rados-wip-yuri3-testing-2021-08-09-1006-distro-basic-smithi/6334693/orig.config.yaml 2021-08-11T15:07:03.056 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-08-11_14:15:52-rados-wip-yuri3-testing-2021-08-09-1006-distro-basic-smithi/6334693 2021-08-11T15:07:03.060 INFO:teuthology.dispatcher.supervisor:Job PID: 19236 2021-08-11T15:07:03.060 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-08-12T03:07:55.036 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-08-12T03:07:55.235 INFO:teuthology.kill:Killing Pids: {19236} 2021-08-12T03:07:56.379 INFO:teuthology.kill:Nuking machines: ['smithi012', 'smithi136'] 2021-08-12T03:07:57.350 INFO:teuthology.kill:2021-08-12 03:07:57,350.350 INFO:teuthology.nuke:targets: 2021-08-12T03:07:57.352 INFO:teuthology.kill: smithi012.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCi7S9/yG2KEv9a6b+dL5I939e21cjAH5tFyAVOXGLHxnZt0EYgqBoXnCWuna0C4AaWJWzA931SSfxgUbw3q6UIJvVluIzXLraFzGDk+9WohjoOU4U0MsRq8Iju4zvo7L24PQqAjpojK5BvKkruGWtWWDLih7RXBP6TY7hD0gEmJhmI4EWinoqQ7RlJ4F0KADhejVIemIJFZNe+nvbdD7NmBtMHvpYa0mAMZrtf+1/90341Rpz3KGbWL2gDfrqSGt5i6xv52VTEJPboir9mzn+Se40pZNPV/ydjCNivZYrHaRmqOngAY6PuPcg5/hY4k0BbtDgZ5QZXe6u2rVvKXKlb+FN+k2mCU1gIE5p7VVswq6C1p6U93uE0u7a2HJXE9IERIS7wKQgsy0sje7HRhvJcMVDc+VSE6Jb6Lea8YLbXPx9/JrKldtm2qBQ3wNkdrbu1UZ3t8TZFnhJKlTVXK+yclWtIgSEclVJrHtRDZ1NqtvD7U/iXeE77xYMmSSGxSG8= 2021-08-12T03:07:57.352 INFO:teuthology.kill: smithi136.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5tZ/2ixQnFZFOEmz4lhgyeKJ0QRx+nSyp+huQYZ6kUHDuPMgMRI+NGzjdSNmvnZka0BlD/n5kBo71OWC+9NLxX4R3uu/sjAk76BG515oM3kvuCfvMf0/LLsghRxFcR35ZUDzvSMPW2LWiYGb6WEPWbSgTu6D1/ATyWgWpy1YrYD8yIFsXSKBmyOI3330TlfHZL7S245hnRsCiCoSaJ1hMa7hWdzx7X31xVwckcMPEbAydmnrk1DG6fFUebpQ9jdvkss5CGCNeo7vcmnLIVWsDCegsuIor72ME3otBAS9NwvtcuPxrL7tjUbViJ5bfuyMwdU6lNV7aFDc2u6xQnLf1JiAgxjRJRx3AigbtDesTM+9ViZ4dHW2+Zv3OOh6batg2vsUf3Kje2N5nF6NEc7YRZlq2AEuu54KMPCxbGyoZqR7gmkCYzy32aIv+99gE2nubyXNZiQQwrVzawhT2Ww9EZKyczosTQOH+e0EcQvXF0X1Bf87JWX5ZTPu2b3vf4Ms= 2021-08-12T03:07:57.353 INFO:teuthology.kill:2021-08-12 03:07:57,350.350 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-12T03:07:57.353 INFO:teuthology.kill:2021-08-12 03:07:57,353.353 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-12T03:07:57.420 INFO:teuthology.kill:2021-08-12 03:07:57,419.419 INFO:teuthology.orchestra.console:Power off smithi012 2021-08-12T03:07:57.435 INFO:teuthology.kill:2021-08-12 03:07:57,435.435 INFO:teuthology.orchestra.console:Power off smithi136 2021-08-12T03:08:05.807 INFO:teuthology.kill:2021-08-12 03:08:05,807.807 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2021-08-12T03:08:09.914 INFO:teuthology.kill:2021-08-12 03:08:09,913.913 INFO:teuthology.orchestra.console:Power off for smithi012 completed 2021-08-12T03:08:10.155 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2021-08-12T03:08:10.156 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2021-08-12T03:08:10.156 INFO:teuthology.misc:Compressing logs... 2021-08-12T03:08:10.157 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-12T03:08:28.491 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-08-12T03:08:28.551 INFO:teuthology.kill:No teuthology processes running 2021-08-12T03:08:29.654 INFO:teuthology.kill:Nuking machines: ['smithi012', 'smithi136'] 2021-08-12T03:08:30.674 INFO:teuthology.kill:2021-08-12 03:08:30,674.674 INFO:teuthology.nuke:targets: 2021-08-12T03:08:30.675 INFO:teuthology.kill: smithi012.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCi7S9/yG2KEv9a6b+dL5I939e21cjAH5tFyAVOXGLHxnZt0EYgqBoXnCWuna0C4AaWJWzA931SSfxgUbw3q6UIJvVluIzXLraFzGDk+9WohjoOU4U0MsRq8Iju4zvo7L24PQqAjpojK5BvKkruGWtWWDLih7RXBP6TY7hD0gEmJhmI4EWinoqQ7RlJ4F0KADhejVIemIJFZNe+nvbdD7NmBtMHvpYa0mAMZrtf+1/90341Rpz3KGbWL2gDfrqSGt5i6xv52VTEJPboir9mzn+Se40pZNPV/ydjCNivZYrHaRmqOngAY6PuPcg5/hY4k0BbtDgZ5QZXe6u2rVvKXKlb+FN+k2mCU1gIE5p7VVswq6C1p6U93uE0u7a2HJXE9IERIS7wKQgsy0sje7HRhvJcMVDc+VSE6Jb6Lea8YLbXPx9/JrKldtm2qBQ3wNkdrbu1UZ3t8TZFnhJKlTVXK+yclWtIgSEclVJrHtRDZ1NqtvD7U/iXeE77xYMmSSGxSG8= 2021-08-12T03:08:30.676 INFO:teuthology.kill: smithi136.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5tZ/2ixQnFZFOEmz4lhgyeKJ0QRx+nSyp+huQYZ6kUHDuPMgMRI+NGzjdSNmvnZka0BlD/n5kBo71OWC+9NLxX4R3uu/sjAk76BG515oM3kvuCfvMf0/LLsghRxFcR35ZUDzvSMPW2LWiYGb6WEPWbSgTu6D1/ATyWgWpy1YrYD8yIFsXSKBmyOI3330TlfHZL7S245hnRsCiCoSaJ1hMa7hWdzx7X31xVwckcMPEbAydmnrk1DG6fFUebpQ9jdvkss5CGCNeo7vcmnLIVWsDCegsuIor72ME3otBAS9NwvtcuPxrL7tjUbViJ5bfuyMwdU6lNV7aFDc2u6xQnLf1JiAgxjRJRx3AigbtDesTM+9ViZ4dHW2+Zv3OOh6batg2vsUf3Kje2N5nF6NEc7YRZlq2AEuu54KMPCxbGyoZqR7gmkCYzy32aIv+99gE2nubyXNZiQQwrVzawhT2Ww9EZKyczosTQOH+e0EcQvXF0X1Bf87JWX5ZTPu2b3vf4Ms= 2021-08-12T03:08:30.696 INFO:teuthology.kill:2021-08-12 03:08:30,695.695 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-12T03:08:30.699 INFO:teuthology.kill:2021-08-12 03:08:30,699.699 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-12T03:08:30.751 INFO:teuthology.kill:2021-08-12 03:08:30,750.750 INFO:teuthology.orchestra.console:Power off smithi012 2021-08-12T03:08:30.764 INFO:teuthology.kill:2021-08-12 03:08:30,764.764 INFO:teuthology.orchestra.console:Power off smithi136 2021-08-12T03:08:30.893 INFO:teuthology.kill:2021-08-12 03:08:30,893.893 INFO:teuthology.orchestra.console:Power off for smithi012 completed 2021-08-12T03:08:30.903 INFO:teuthology.kill:2021-08-12 03:08:30,903.903 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2021-08-12T03:08:31.037 INFO:teuthology.kill:2021-08-12 03:08:31,036.036 INFO:teuthology.lock.ops:unlocked smithi012.front.sepia.ceph.com 2021-08-12T03:08:31.041 INFO:teuthology.kill:2021-08-12 03:08:31,040.040 INFO:teuthology.lock.ops:unlocked smithi136.front.sepia.ceph.com 2021-08-12T03:10:31.325 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-08-12T03:10:31.410 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-08-12T03:10:31.411 INFO:teuthology.nuke:Checking targets against current locks 2021-08-12T03:10:31.499 INFO:teuthology.nuke:Not nuking smithi012.front.sepia.ceph.com because description doesn't match 2021-08-12T03:10:31.500 INFO:teuthology.nuke:Not nuking smithi136.front.sepia.ceph.com because description doesn't match