2023-07-26T02:05:56.206 INFO:root:teuthology version: 0.0.1.dev160+g407880c 2023-07-26T02:05:56.207 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-26T02:05:56.208 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-26T02:06:10.080 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7c011b7430>, signals=[15]) 2023-07-26T02:06:10.083 INFO:teuthology.lock.ops:Start node 'smithi137.front.sepia.ceph.com' reimaging 2023-07-26T02:06:10.083 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: reset os type and version on server 2023-07-26T02:06:10.084 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2023-07-26T02:06:10.108 INFO:teuthology.lock.ops:Start node 'smithi188.front.sepia.ceph.com' reimaging 2023-07-26T02:06:10.108 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: reset os type and version on server 2023-07-26T02:06:10.109 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2023-07-26T02:06:10.110 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /host {"name": "smithi137"} 2023-07-26T02:06:15.149 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /host {"name": "smithi188"} 2023-07-26T02:06:20.575 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-07-26T02:06:20.607 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-07-26T02:06:20.628 DEBUG:teuthology.provision.fog.smithi137:Requesting image smithi_rhel_8.6 (ID 5) 2023-07-26T02:06:20.629 DEBUG:teuthology.provision.fog.smithi137:FOG request: PUT /host/242 {"imageID": 5} 2023-07-26T02:06:20.660 DEBUG:teuthology.provision.fog.smithi188:Requesting image smithi_rhel_8.6 (ID 5) 2023-07-26T02:06:20.660 DEBUG:teuthology.provision.fog.smithi188:FOG request: PUT /host/293 {"imageID": 5} 2023-07-26T02:06:20.683 INFO:teuthology.provision.fog.smithi137:Scheduling deploy of rhel 8.6 2023-07-26T02:06:20.683 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-07-26T02:06:20.708 INFO:teuthology.provision.fog.smithi188:Scheduling deploy of rhel 8.6 2023-07-26T02:06:20.708 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-07-26T02:06:20.745 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /tasktype {"name": "deploy"} 2023-07-26T02:06:30.754 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /tasktype {"name": "deploy"} 2023-07-26T02:06:35.803 DEBUG:teuthology.provision.fog.smithi188:FOG request: POST /host/293/task {"taskTypeID": 1} 2023-07-26T02:06:40.809 DEBUG:teuthology.provision.fog.smithi137:FOG request: POST /host/242/task {"taskTypeID": 1} 2023-07-26T02:06:41.103 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-07-26T02:06:41.108 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-07-26T02:06:46.784 INFO:teuthology.orchestra.console:Power off smithi188 2023-07-26T02:06:46.785 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-26T02:06:46.814 INFO:teuthology.orchestra.console:Power off smithi137 2023-07-26T02:06:46.815 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-26T02:06:46.825 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-26T02:06:46.825 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:06:46.837 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-26T02:06:46.838 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:06:46.848 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-26T02:06:46.859 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-26T02:06:46.950 INFO:teuthology.orchestra.console:Power off for smithi188 completed 2023-07-26T02:06:46.961 INFO:teuthology.orchestra.console:Power off for smithi137 completed 2023-07-26T02:06:47.052 INFO:teuthology.orchestra.console:Power on smithi188 2023-07-26T02:06:47.052 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-26T02:06:47.063 INFO:teuthology.orchestra.console:Power on smithi137 2023-07-26T02:06:47.063 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-26T02:06:47.075 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-26T02:06:47.076 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:06:47.086 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-26T02:06:47.087 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:06:47.098 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-26T02:06:47.110 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-26T02:06:51.100 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:06:51.111 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:06:52.210 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:06:52.222 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:06:56.213 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:06:56.222 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:06:57.324 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:06:57.334 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:01.328 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:01.338 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:02.440 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:02.449 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:06.442 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:06.451 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:07.554 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:07.562 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:11.555 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:11.563 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:12.666 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:12.674 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:16.671 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:16.680 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:17.783 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:17.791 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:21.784 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:21.793 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:22.896 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:22.903 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:26.898 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:26.908 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:28.010 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:28.018 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:32.015 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:32.024 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:33.127 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:33.134 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:37.130 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:37.141 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:38.243 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:38.252 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:42.245 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:42.253 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:42.355 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-26T02:07:42.457 INFO:teuthology.orchestra.console:Power on for smithi188 completed 2023-07-26T02:07:42.558 INFO:teuthology.provision.fog.smithi188:Waiting for deploy to finish 2023-07-26T02:07:42.562 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-07-26T02:07:43.363 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:47.364 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:48.475 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:52.477 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:53.588 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:07:57.592 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:07:57.643 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-07-26T02:08:00.501 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:08:00.501 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds 2023-07-26T02:08:00.502 DEBUG:teuthology.provision.fog.smithi137:Canceling deploy task with ID None 2023-07-26T02:08:00.502 DEBUG:teuthology.provision.fog.smithi137:FOG request: DELETE /task/cancel {"id": None} 2023-07-26T02:08:05.581 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 85, in create self.remote.console.power_on() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/orchestra/console.py", line 252, in power_on raise RuntimeError(err_msg) RuntimeError: Failed to power on smithi137 2023-07-26T02:08:05.581 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-26T02:08:05.582 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-26T02:08:10.690 INFO:teuthology.nuke:Checking targets against current locks 2023-07-26T02:08:15.698 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-07-26T02:08:25.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T02:08:25.767 DEBUG:teuthology.nuke:shortname: smithi137 2023-07-26T02:08:25.767 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-26T02:08:31.771 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': None, 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': False, 'locked_since': None, 'locked_by': None, 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-07-26T02:08:31.772 ERROR:teuthology.nuke:Could not nuke {'smithi137.front.sepia.ceph.com': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/nuke/__init__.py", line 294, in nuke_one nuke_helper(ctx, should_unlock, keep_logs, should_reboot) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/nuke/__init__.py", line 318, in nuke_helper check_lock.check_lock(ctx, None, check_up=False) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/task/internal/check_lock.py", line 28, in check_lock assert status['locked'], \ AssertionError: machine smithi137.front.sepia.ceph.com is not locked 2023-07-26T02:08:31.773 DEBUG:teuthology.nuke:shortname: smithi188 2023-07-26T02:08:31.773 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-26T02:08:31.776 ERROR:teuthology.orchestra.connection:Error authenticating with smithi188.front.sepia.ceph.com: Authentication failed. 2023-07-26T02:08:31.777 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-26T02:08:31.778 DEBUG:teuthology.orchestra.run.smithi188:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-26T02:08:31.779 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T02:08:31.797 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-25_15:03:30-fs-reef-release-distro-default-smithi/7351530', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-26 02:05:45.360780', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMIsddPZ/WxrvYSki7UwnO6iNRDoeWehVS15YXx+jJjcpP0efFozBaJSenQ0gBeFMLF1Nd6b7pKVSIyFlvULlNw='} 2023-07-26T02:08:31.852 INFO:teuthology.orchestra.console:Power off smithi188 2023-07-26T02:08:31.852 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-26T02:08:31.874 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-26T02:08:31.875 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:08:31.899 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-26T02:08:32.789 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T02:08:33.796 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T02:08:34.802 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T02:08:35.808 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T02:08:35.901 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:08:36.013 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-26T02:08:36.816 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T02:08:37.822 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T02:08:38.828 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T02:08:39.833 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T02:08:40.015 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:08:41.127 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:08:45.129 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:08:46.241 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:08:50.244 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:08:51.356 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:08:55.358 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:08:56.470 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:09:00.473 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:09:01.583 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T02:09:05.585 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T02:09:05.695 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-26T02:09:05.797 INFO:teuthology.orchestra.console:Power off for smithi188 completed 2023-07-26T02:09:15.968 INFO:teuthology.lock.ops:unlocked: smithi188.front.sepia.ceph.com 2023-07-26T02:09:15.969 ERROR:teuthology.nuke:Could not nuke the following targets: targets: smithi137.front.sepia.ceph.com: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw 2023-07-26T02:09:25.466 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 85, in create self.remote.console.power_on() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/orchestra/console.py", line 252, in power_on raise RuntimeError(err_msg) RuntimeError: Failed to power on smithi137