2023-05-25T22:15:41.433 INFO:root:teuthology version: 0.0.1.dev106+ga48c8bd 2023-05-25T22:15:41.646 INFO:teuthology.lock.ops:Start node 'smithi033.front.sepia.ceph.com' reimaging 2023-05-25T22:15:41.647 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: reset os type and version on server 2023-05-25T22:15:41.647 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2023-05-25T22:15:41.673 INFO:teuthology.lock.ops:Node 'smithi033.front.sepia.ceph.com' reimaging is complete 2023-05-25T22:15:41.673 INFO:teuthology.lock.ops:Start node 'smithi146.front.sepia.ceph.com' reimaging 2023-05-25T22:15:41.673 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: reset os type and version on server 2023-05-25T22:15:41.673 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2023-05-25T22:15:41.696 INFO:teuthology.lock.ops:Node 'smithi146.front.sepia.ceph.com' reimaging is complete 2023-05-25T22:15:41.697 INFO:teuthology.lock.ops:Start node 'smithi203.front.sepia.ceph.com' reimaging 2023-05-25T22:15:41.697 INFO:teuthology.lock.ops:Updating [smithi203.front.sepia.ceph.com]: reset os type and version on server 2023-05-25T22:15:41.697 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2023-05-25T22:15:41.713 INFO:teuthology.lock.ops:Node 'smithi203.front.sepia.ceph.com' reimaging is complete 2023-05-25T22:15:41.807 INFO:teuthology.provision.fog.smithi033:Scheduling deploy of ubuntu 22.04 2023-05-25T22:15:41.836 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of ubuntu 22.04 2023-05-25T22:15:41.856 INFO:teuthology.provision.fog.smithi203:Scheduling deploy of ubuntu 22.04 2023-05-25T22:15:42.053 INFO:teuthology.orchestra.console:Power off smithi033 2023-05-25T22:15:42.071 INFO:teuthology.orchestra.console:Power off smithi146 2023-05-25T22:15:42.092 INFO:teuthology.orchestra.console:Power off smithi203 2023-05-25T22:15:42.203 INFO:teuthology.orchestra.console:Power off for smithi033 completed 2023-05-25T22:15:42.304 INFO:teuthology.orchestra.console:Power on smithi033 2023-05-25T22:15:50.461 INFO:teuthology.orchestra.console:Power off for smithi203 completed 2023-05-25T22:15:50.563 INFO:teuthology.orchestra.console:Power on smithi203 2023-05-25T22:15:50.692 INFO:teuthology.orchestra.console:Power on for smithi033 completed 2023-05-25T22:15:50.793 INFO:teuthology.provision.fog.smithi033:Waiting for deploy to finish 2023-05-25T22:15:54.565 INFO:teuthology.orchestra.console:Power off for smithi146 completed 2023-05-25T22:15:54.666 INFO:teuthology.orchestra.console:Power on smithi146 2023-05-25T22:15:58.922 INFO:teuthology.orchestra.console:Power on for smithi146 completed 2023-05-25T22:15:58.928 INFO:teuthology.orchestra.console:Power on for smithi203 completed 2023-05-25T22:15:59.023 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish 2023-05-25T22:15:59.029 INFO:teuthology.provision.fog.smithi203:Waiting for deploy to finish 2023-05-25T22:19:41.040 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-05-25T22:19:50.096 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-05-25T22:19:59.184 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-05-25T22:20:04.240 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-05-25T22:20:13.296 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-05-25T22:20:19.696 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.203 2023-05-25T22:20:22.384 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-05-25T22:20:28.752 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.203 2023-05-25T22:20:31.440 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-05-25T22:20:36.596 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-05-25T22:20:37.808 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.203 2023-05-25T22:20:40.500 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-05-25T22:20:42.601 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-05-25T22:20:46.864 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.203 2023-05-25T22:20:48.605 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-05-25T22:20:49.584 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-05-25T22:20:55.920 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.203 2023-05-25T22:20:58.640 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-05-25T22:21:00.720 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-05-25T22:21:06.724 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-05-25T22:21:12.990 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-05-25T22:21:13.462 INFO:teuthology.provision.fog.smithi033:Node is ready 2023-05-25T22:21:13.509 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033.front.sepia.ceph.com 2023-05-25T22:21:13.557 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33 smithi033.front.sepia.ceph.com smithi033 2023-05-25T22:21:13.877 INFO:teuthology.provision.fog.smithi033:Deploy complete! 2023-05-25T22:21:13.877 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'smithi033.front.sepia.ceph.com' 2023-05-25T22:21:13.962 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed. 2023-05-25T22:21:15.043 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed. 2023-05-25T22:21:16.126 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed. 2023-05-25T22:21:17.210 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed. 2023-05-25T22:21:18.294 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed. 2023-05-25T22:21:19.378 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed. 2023-05-25T22:21:20.368 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.203 2023-05-25T22:21:20.462 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed. 2023-05-25T22:21:21.547 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed. 2023-05-25T22:21:22.631 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed. 2023-05-25T22:21:23.715 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed. 2023-05-25T22:21:23.716 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/dispatcher/supervisor.py", line 226, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/lock/ops.py", line 324, in reimage_machines log.info("Node '%s' reimaging is complete", machine) File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/provision/__init__.py", line 42, in reimage if remote_.os.name.lower() != os_type.lower() or remote_.os.version != os_version: File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/orchestra/remote.py", line 322, in os os_release = self.sh('cat /etc/os-release').strip() File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/orchestra/remote.py", line 524, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi033 2023-05-25T22:21:23.829 INFO:teuthology.nuke:Checking targets against current locks 2023-05-25T22:21:23.882 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-05-25T22:21:23.916 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-05-25T22:21:23.931 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-05-25T22:21:23.959 INFO:teuthology.orchestra.console:Power off smithi033 2023-05-25T22:21:24.002 INFO:teuthology.orchestra.console:Power off smithi146 2023-05-25T22:21:24.039 INFO:teuthology.orchestra.console:Power off smithi203 2023-05-25T22:21:29.424 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.203 2023-05-25T22:21:32.333 INFO:teuthology.orchestra.console:Power off for smithi033 completed 2023-05-25T22:21:32.399 INFO:teuthology.orchestra.console:Power off for smithi203 completed 2023-05-25T22:21:32.489 INFO:teuthology.lock.ops:unlocked: smithi033.front.sepia.ceph.com 2023-05-25T22:21:32.553 INFO:teuthology.lock.ops:unlocked: smithi203.front.sepia.ceph.com 2023-05-25T22:21:36.478 INFO:teuthology.orchestra.console:Power off for smithi146 completed 2023-05-25T22:21:36.637 INFO:teuthology.lock.ops:unlocked: smithi146.front.sepia.ceph.com 2023-05-25T22:21:37.047 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/dispatcher/supervisor.py", line 56, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/dispatcher/supervisor.py", line 226, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/lock/ops.py", line 324, in reimage_machines log.info("Node '%s' reimaging is complete", machine) File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/provision/__init__.py", line 42, in reimage if remote_.os.name.lower() != os_type.lower() or remote_.os.version != os_version: File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/orchestra/remote.py", line 322, in os os_release = self.sh('cat /etc/os-release').strip() File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_a48c8bd9a62aa04add18667dc7429d554b46faa2/teuthology/orchestra/remote.py", line 524, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi033