2021-04-09T18:31:04.703 INFO:root:teuthology version: 1.1.0-85d61eae 2021-04-09T18:31:04.810 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi019.front.sepia.ceph.com' reimaging 2021-04-09T18:31:04.811 INFO:teuthology.lock.ops:Updating [ubuntu@smithi019.front.sepia.ceph.com]: reset os type and version on server 2021-04-09T18:31:04.812 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2021-04-09T18:31:04.831 INFO:teuthology.lock.ops:Node 'ubuntu@smithi019.front.sepia.ceph.com' reimaging is complete 2021-04-09T18:31:04.834 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi115.front.sepia.ceph.com' reimaging 2021-04-09T18:31:04.835 INFO:teuthology.lock.ops:Updating [ubuntu@smithi115.front.sepia.ceph.com]: reset os type and version on server 2021-04-09T18:31:04.835 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2021-04-09T18:31:04.854 INFO:teuthology.lock.ops:Node 'ubuntu@smithi115.front.sepia.ceph.com' reimaging is complete 2021-04-09T18:31:05.057 INFO:teuthology.provision.fog.smithi019:Scheduling deploy of rhel 7.9 2021-04-09T18:31:05.077 INFO:teuthology.provision.fog.smithi115:Scheduling deploy of rhel 7.9 2021-04-09T18:31:05.461 INFO:teuthology.orchestra.console:Power off smithi019 2021-04-09T18:31:05.543 INFO:teuthology.orchestra.console:Power off smithi115 2021-04-09T18:31:13.839 INFO:teuthology.orchestra.console:Power off for smithi019 completed 2021-04-09T18:31:13.940 INFO:teuthology.orchestra.console:Power on smithi019 2021-04-09T18:31:18.028 INFO:teuthology.orchestra.console:Power off for smithi115 completed 2021-04-09T18:31:18.130 INFO:teuthology.orchestra.console:Power on smithi115 2021-04-09T18:31:22.320 INFO:teuthology.orchestra.console:Power on for smithi019 completed 2021-04-09T18:31:22.382 INFO:teuthology.orchestra.console:Power on for smithi115 completed 2021-04-09T18:31:22.422 INFO:teuthology.provision.fog.smithi019:Waiting for deploy to finish 2021-04-09T18:31:22.484 INFO:teuthology.provision.fog.smithi115:Waiting for deploy to finish 2021-04-09T18:34:09.421 ERROR:teuthology.orchestra.connection:Error authenticating with smithi115.front.sepia.ceph.com: Authentication failed. 2021-04-09T18:35:45.776 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-04-09T18:35:46.200 INFO:teuthology.provision.fog.smithi019:Node is ready 2021-04-09T18:35:46.263 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019.front.sepia.ceph.com 2021-04-09T18:35:46.306 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19 smithi019.front.sepia.ceph.com smithi019 2021-04-09T18:35:46.676 INFO:teuthology.provision.fog.smithi019:Deploy complete! 2021-04-09T18:35:54.293 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-04-09T18:36:14.802 INFO:teuthology.provision.fog.smithi115:Node is ready 2021-04-09T18:36:14.868 INFO:teuthology.orchestra.run.smithi115.stdout:smithi162 2021-04-09T18:36:14.909 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.162 smithi162.front.sepia.ceph.com smithi162 2021-04-09T18:36:15.275 INFO:teuthology.provision.fog.smithi115:Deploy complete! 2021-04-09T18:36:15.365 ERROR:teuthology.misc:write (smithi019.front.sepia.ceph.com): Connection refused 2021-04-09T18:36:16.373 ERROR:teuthology.misc:write (smithi019.front.sepia.ceph.com): Connection refused 2021-04-09T18:36:17.378 ERROR:teuthology.misc:write (smithi019.front.sepia.ceph.com): Connection refused 2021-04-09T18:36:18.385 ERROR:teuthology.misc:write (smithi019.front.sepia.ceph.com): Connection refused 2021-04-09T18:36:20.392 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/dispatcher/supervisor.py", line 163, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/lock/ops.py", line 318, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/lock/ops.py", line 285, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/misc.py", line 1103, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan smithi019.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds 2021-04-09T18:36:20.654 INFO:teuthology.nuke:Checking targets against current locks 2021-04-09T18:36:20.988 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-04-09T18:36:21.012 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-04-09T18:36:26.108 INFO:teuthology.orchestra.console:Power off smithi019 2021-04-09T18:36:26.173 INFO:teuthology.orchestra.console:Power off smithi115 2021-04-09T18:36:26.284 INFO:teuthology.orchestra.console:Power off for smithi019 completed 2021-04-09T18:36:26.368 INFO:teuthology.orchestra.console:Power off for smithi115 completed 2021-04-09T18:36:26.753 ERROR:teuthology.lock.ops:failed to unlock smithi019.front.sepia.ceph.com. reason: Cannot unlock an already-unlocked node 2021-04-09T18:36:26.796 INFO:teuthology.lock.ops:unlocked smithi115.front.sepia.ceph.com 2021-04-09T18:36:26.797 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/dispatcher/supervisor.py", line 50, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/dispatcher/supervisor.py", line 163, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/lock/ops.py", line 318, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/lock/ops.py", line 285, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/misc.py", line 1103, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan smithi019.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds