2023-07-17T18:18:29.462 INFO:root:teuthology version: 0.0.1.dev147+gff9991b 2023-07-17T18:18:29.463 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-17T18:18:29.463 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-17T18:18:39.570 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0309b4d8b0>, signals=[15]) 2023-07-17T18:18:39.571 INFO:teuthology.lock.ops:Start node 'smithi136.front.sepia.ceph.com' reimaging 2023-07-17T18:18:39.571 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: reset os type and version on server 2023-07-17T18:18:39.572 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2023-07-17T18:18:46.569 INFO:teuthology.lock.ops:Start node 'smithi155.front.sepia.ceph.com' reimaging 2023-07-17T18:18:46.570 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: reset os type and version on server 2023-07-17T18:18:46.570 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2023-07-17T18:18:46.572 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /host {"name": "smithi136"} 2023-07-17T18:19:01.567 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /host {"name": "smithi155"} 2023-07-17T18:19:11.627 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-07-17T18:19:16.634 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-07-17T18:19:21.686 DEBUG:teuthology.provision.fog.smithi155:Requesting image smithi_rhel_8.6 (ID 5) 2023-07-17T18:19:21.686 DEBUG:teuthology.provision.fog.smithi155:FOG request: PUT /host/260 {"imageID": 5} 2023-07-17T18:19:26.695 DEBUG:teuthology.provision.fog.smithi136:Requesting image smithi_rhel_8.6 (ID 5) 2023-07-17T18:19:26.695 DEBUG:teuthology.provision.fog.smithi136:FOG request: PUT /host/241 {"imageID": 5} 2023-07-17T18:19:31.749 INFO:teuthology.provision.fog.smithi136:Scheduling deploy of rhel 8.6 2023-07-17T18:19:31.749 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-07-17T18:19:36.749 INFO:teuthology.provision.fog.smithi155:Scheduling deploy of rhel 8.6 2023-07-17T18:19:36.749 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-07-17T18:19:36.844 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /tasktype {"name": "deploy"} 2023-07-17T18:19:36.849 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /tasktype {"name": "deploy"} 2023-07-17T18:19:36.884 DEBUG:teuthology.provision.fog.smithi136:FOG request: POST /host/241/task {"taskTypeID": 1} 2023-07-17T18:19:36.888 DEBUG:teuthology.provision.fog.smithi155:FOG request: POST /host/260/task {"taskTypeID": 1} 2023-07-17T18:19:36.939 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-07-17T18:19:36.942 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-07-17T18:19:47.079 INFO:teuthology.orchestra.console:Power off smithi136 2023-07-17T18:19:47.080 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-17T18:19:47.099 INFO:teuthology.orchestra.console:Power off smithi155 2023-07-17T18:19:47.100 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-17T18:19:47.112 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:19:47.128 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:19:51.144 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:19:51.157 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:19:56.257 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:19:56.267 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:01.372 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:01.380 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:06.488 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:06.498 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:11.604 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:11.615 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:16.718 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:16.727 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:21.829 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:21.839 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:22.050 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2023-07-17T18:20:22.152 INFO:teuthology.orchestra.console:Power on smithi155 2023-07-17T18:20:22.152 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-17T18:20:22.173 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:26.200 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:26.941 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:27.154 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2023-07-17T18:20:27.256 INFO:teuthology.orchestra.console:Power on smithi136 2023-07-17T18:20:27.256 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-17T18:20:27.277 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:31.301 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:31.311 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:31.513 INFO:teuthology.orchestra.console:Power on for smithi136 completed 2023-07-17T18:20:31.614 INFO:teuthology.provision.fog.smithi136:Waiting for deploy to finish 2023-07-17T18:20:31.615 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-07-17T18:20:31.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T18:20:34.768 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-07-17T18:20:36.424 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:40.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T18:20:45.778 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:48.852 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-07-17T18:20:50.890 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:54.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T18:20:56.002 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:20:57.940 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-07-17T18:21:01.113 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:21:03.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T18:21:06.227 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:21:07.024 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-07-17T18:21:11.339 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:21:13.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T18:21:18.034 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:21:21.104 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-07-17T18:21:23.147 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:21:27.109 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T18:21:28.260 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:21:30.160 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-07-17T18:21:33.376 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:21:36.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T18:21:41.169 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:21:42.282 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds 2023-07-17T18:21:42.283 DEBUG:teuthology.provision.fog.smithi155:Canceling deploy task with ID 392236 2023-07-17T18:21:42.284 DEBUG:teuthology.provision.fog.smithi155:FOG request: DELETE /task/cancel {"id": 392236} 2023-07-17T18:21:52.425 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/fog.py", line 85, in create self.remote.console.power_on() File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/console.py", line 243, in power_on raise RuntimeError(err_msg) RuntimeError: Failed to power on smithi155 2023-07-17T18:21:52.426 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-17T18:21:52.426 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-17T18:21:52.519 INFO:teuthology.nuke:Checking targets against current locks 2023-07-17T18:21:52.581 DEBUG:teuthology.nuke:shortname: smithi136 2023-07-17T18:21:52.581 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-17T18:21:52.606 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-17_14:37:31-rados-wip-yuri-testing-2023-07-14-1641-distro-default-smithi/7341655', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-17 18:18:23.624474', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDG9M6sIVM8ADB9qplps8/94/wvM1MRCKBY0AY3Ggspg/gLo1zyttKpJMdxIFlsXgIhQm3aDJ7NQAv6SLNKodDzUOzK7lqBmSnYfFGfs+2zB2qwBj7dIbso9lzzPernEEzn+VVBN9ux90iWXIXOLkl2sEp437YJxtosxEA7wVIbD8Dbf/Uofl/EB7IeFunM2q2eOY3+RZIY9wD0lLW2v5fXkFX34t2Z1WUN3ERj+o2j50eXww3yp+Ak9zQpAAARQL3DnDtu79ZTmf3LSlgRyPCmK1vX4bAviKxRhike3hj4NvOdQsikLcBMp1TRImzQqGpevA3PGIeU0MJ7WiQ/y9zQSfCFp4iCTdiWkU7PZaQp1wo5EBu59I0uu17kF5roVMcBPpPao5ojlyCgxyWD8kxnkxJw7s1olJvrfyDKeKF1AEPo1bK06FFzxtVXvoXpK+iHD32niFemkkqnBRHHxW8AF6icbkVepUgTRtOiXJxoHa8nbDRWcwFuJd6JkiFNd9c='} 2023-07-17T18:21:52.608 DEBUG:teuthology.nuke:shortname: smithi155 2023-07-17T18:21:52.609 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-17T18:21:52.633 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-17_14:37:31-rados-wip-yuri-testing-2023-07-14-1641-distro-default-smithi/7341655', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-17 18:18:23.622403', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCsr+P7lbRG+t954kdAI8Sopbl8nIp4rtxlCddiOO7g4qJZVxwRg4+m8W77FCWYU72vYQe8ucMlOO2JSazV/pik='} 2023-07-17T18:21:52.662 INFO:teuthology.orchestra.console:Power off smithi136 2023-07-17T18:21:52.663 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-17T18:21:52.684 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:21:52.704 INFO:teuthology.orchestra.console:Power off smithi155 2023-07-17T18:21:52.705 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-17T18:21:52.730 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:21:56.720 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:21:56.756 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:22:01.831 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:22:01.867 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:22:06.943 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:22:06.984 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:22:07.155 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2023-07-17T18:22:17.296 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:22:17.340 INFO:teuthology.lock.ops:unlocked: smithi136.front.sepia.ceph.com 2023-07-17T18:22:22.408 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:22:27.520 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:22:32.636 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:22:37.749 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:22:41.172 WARNING:teuthology.provision.fog.smithi136:timed out 2023-07-17T18:22:42.860 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:22:47.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T18:22:47.971 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:22:50.264 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-07-17T18:22:53.084 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:22:56.269 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T18:22:58.196 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T18:22:58.407 INFO:teuthology.orchestra.console:Power off for smithi155 completed 2023-07-17T18:23:03.513 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-07-17T18:23:07.023 INFO:teuthology.lock.ops:unlocked: smithi155.front.sepia.ceph.com 2023-07-17T18:23:07.275 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/fog.py", line 85, in create self.remote.console.power_on() File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/console.py", line 243, in power_on raise RuntimeError(err_msg) RuntimeError: Failed to power on smithi155