2023-09-21T12:36:24.430 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-09-21T12:36:24.432 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-21T12:36:24.580 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f2f8e54ec70>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f2f8e2f8ca0>, signals=[15])
2023-09-21T12:36:24.581 INFO:teuthology.lock.ops:Start node 'smithi104.front.sepia.ceph.com' reimaging
2023-09-21T12:36:24.582 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: reset os type and version on server
2023-09-21T12:36:24.582 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server
2023-09-21T12:36:24.602 INFO:teuthology.lock.ops:Start node 'smithi169.front.sepia.ceph.com' reimaging
2023-09-21T12:36:24.602 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: reset os type and version on server
2023-09-21T12:36:24.603 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server
2023-09-21T12:36:24.604 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /host {"name": "smithi104"}
2023-09-21T12:36:24.618 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /host {"name": "smithi169"}
2023-09-21T12:36:24.689 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-09-21T12:36:24.692 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-09-21T12:36:24.729 DEBUG:teuthology.provision.fog.smithi104:Requesting image smithi_rhel_8.6 (ID 5)
2023-09-21T12:36:24.729 DEBUG:teuthology.provision.fog.smithi104:FOG request: PUT /host/209 {"imageID": 5}
2023-09-21T12:36:24.733 DEBUG:teuthology.provision.fog.smithi169:Requesting image smithi_rhel_8.6 (ID 5)
2023-09-21T12:36:24.733 DEBUG:teuthology.provision.fog.smithi169:FOG request: PUT /host/274 {"imageID": 5}
2023-09-21T12:36:24.777 INFO:teuthology.provision.fog.smithi104:Scheduling deploy of rhel 8.6
2023-09-21T12:36:24.777 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-09-21T12:36:24.787 INFO:teuthology.provision.fog.smithi169:Scheduling deploy of rhel 8.6
2023-09-21T12:36:24.787 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-09-21T12:36:24.859 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /tasktype {"name": "deploy"}
2023-09-21T12:36:24.862 DEBUG:teuthology.provision.fog.smithi169:Canceling deploy task with ID 499563
2023-09-21T12:36:24.863 DEBUG:teuthology.provision.fog.smithi169:FOG request: DELETE /task/cancel {"id": 499563}
2023-09-21T12:36:24.899 DEBUG:teuthology.provision.fog.smithi104:FOG request: POST /host/209/task {"taskTypeID": 1}
2023-09-21T12:36:24.911 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /tasktype {"name": "deploy"}
2023-09-21T12:36:24.956 DEBUG:teuthology.provision.fog.smithi169:FOG request: POST /host/274/task {"taskTypeID": 1}
2023-09-21T12:36:24.968 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-09-21T12:36:25.036 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-09-21T12:36:25.100 INFO:teuthology.orchestra.console.smithi104:Power off
2023-09-21T12:36:25.101 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:36:25.137 INFO:teuthology.orchestra.console.smithi169:Power off
2023-09-21T12:36:25.138 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:36:25.159 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off
2023-09-21T12:36:25.159 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:25.181 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on
2023-09-21T12:36:29.183 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:29.291 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:36:29.392 INFO:teuthology.orchestra.console.smithi169:Power off completed
2023-09-21T12:36:29.493 INFO:teuthology.orchestra.console.smithi169:Power on
2023-09-21T12:36:29.493 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:36:29.515 DEBUG:teuthology.orchestra.console.smithi169:power on output: Chassis Power Control: Up/On
2023-09-21T12:36:29.515 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:29.536 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:36:33.538 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:33.647 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:36:37.652 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:37.761 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:36:41.762 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:41.870 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:36:45.171 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:36:45.172 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:36:45.873 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:45.982 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:36:49.986 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:50.096 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:36:54.098 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:54.207 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:36:58.213 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:58.323 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:37:02.327 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:02.437 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:37:05.240 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:37:05.240 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:37:06.440 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:06.549 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:37:10.553 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:10.662 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:37:14.666 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:14.774 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:37:18.778 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:18.886 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:37:22.891 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:22.999 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:37:25.311 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:37:25.312 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:26.320 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:37:27.001 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:27.110 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:37:27.111 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T12:37:27.211 DEBUG:teuthology.provision.fog.smithi169:Canceling deploy task with ID 499565
2023-09-21T12:37:27.212 DEBUG:teuthology.provision.fog.smithi169:FOG request: DELETE /task/cancel {"id": 499565}
2023-09-21T12:37:27.271 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi169
2023-09-21T12:37:27.271 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-21T12:37:27.415 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=9a0dcf4e8db84d418027ffeb3a25de28
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi169
2023-09-21T12:37:27.483 INFO:teuthology.nuke:Checking targets against current locks
2023-09-21T12:37:27.548 DEBUG:teuthology.nuke:shortname: smithi104
2023-09-21T12:37:27.548 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-21T12:37:27.574 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-09-20_16:32:45-fs-wip-lusov-config-format-distro-default-smithi/7399885', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 12:36:23.663578', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}
2023-09-21T12:37:27.577 DEBUG:teuthology.nuke:shortname: smithi169
2023-09-21T12:37:27.578 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-21T12:37:27.601 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-09-20_16:32:45-fs-wip-lusov-config-format-distro-default-smithi/7399885', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 12:36:23.665223', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGQUYHLq1eI9PL2rZITOjj7VTyBjXQIT3Kl0Qgzvf8KQznGTY002n4wXGRdf6es+9q9ZLymVamnLq99Bx8pkHWk='}
2023-09-21T12:37:27.629 INFO:teuthology.orchestra.console.smithi104:Power off
2023-09-21T12:37:27.630 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:37:27.670 INFO:teuthology.orchestra.console.smithi169:Power off
2023-09-21T12:37:27.671 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:37:27.694 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off
2023-09-21T12:37:27.695 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:27.716 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off
2023-09-21T12:37:27.817 INFO:teuthology.orchestra.console.smithi169:Power off completed
2023-09-21T12:37:27.965 INFO:teuthology.lock.ops:unlocked: smithi169.front.sepia.ceph.com
2023-09-21T12:37:30.323 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:31.431 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:37:35.436 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:36.545 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:37:40.550 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:41.659 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:37:45.664 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:46.772 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:37:47.702 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:37:47.702 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:37:50.774 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:51.882 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:37:55.887 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:56.997 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:38:01.002 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:02.110 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:38:06.115 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:07.224 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:38:07.774 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:38:07.775 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:38:11.229 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:12.338 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:38:16.342 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:17.452 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:38:21.457 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:22.567 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:38:26.570 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:27.679 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:38:27.843 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:38:27.844 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:28.852 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:38:31.683 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:32.793 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:38:32.854 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:33.963 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:38:36.798 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:37.908 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:38:37.909 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T12:38:37.909 ERROR:teuthology.orchestra.console.smithi104:Failed to power off
2023-09-21T12:38:37.965 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:38.013 INFO:teuthology.orchestra.console.smithi104:Power on
2023-09-21T12:38:38.014 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:38:39.076 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:38:43.078 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:44.186 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:38:48.190 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:49.299 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:38:53.302 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:54.412 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:38:58.083 DEBUG:teuthology.orchestra.console.smithi104:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:38:58.084 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:38:58.413 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:59.524 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:39:03.529 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:04.639 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:39:08.644 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:09.754 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:39:13.758 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:14.867 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:39:18.157 DEBUG:teuthology.orchestra.console.smithi104:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:39:18.157 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:39:18.870 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:19.979 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:39:23.982 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:25.092 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:39:29.097 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:30.206 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:39:34.210 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:35.319 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:39:38.227 DEBUG:teuthology.orchestra.console.smithi104:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:39:38.228 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:39.237 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:39:39.321 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:40.430 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T12:39:40.431 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T12:39:40.431 ERROR:teuthology.orchestra.console.smithi104:Failed to power off
2023-09-21T12:39:40.579 INFO:teuthology.lock.ops:unlocked: smithi104.front.sepia.ceph.com
2023-09-21T12:39:40.824 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi169