2023-08-15T22:55:21.767 INFO:root:teuthology version: 0.0.1.dev183+g5435569
2023-08-15T22:55:21.768 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-15T22:55:31.605 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f1d9ea3f790>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f1d9e31df70>, signals=[15])
2023-08-15T22:55:31.608 INFO:teuthology.lock.ops:Start node 'smithi002.front.sepia.ceph.com' reimaging
2023-08-15T22:55:31.609 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: reset os type and version on server
2023-08-15T22:55:31.609 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server
2023-08-15T22:55:31.633 INFO:teuthology.lock.ops:Start node 'smithi154.front.sepia.ceph.com' reimaging
2023-08-15T22:55:31.634 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: reset os type and version on server
2023-08-15T22:55:31.634 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server
2023-08-15T22:55:31.635 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /host {"name": "smithi002"}
2023-08-15T22:55:41.666 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /host {"name": "smithi154"}
2023-08-15T22:55:46.693 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /image {"name": "smithi_ubuntu_20.04"}
2023-08-15T22:55:56.729 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /image {"name": "smithi_ubuntu_20.04"}
2023-08-15T22:56:01.737 DEBUG:teuthology.provision.fog.smithi002:Requesting image smithi_ubuntu_20.04 (ID 2)
2023-08-15T22:56:01.737 DEBUG:teuthology.provision.fog.smithi002:FOG request: PUT /host/108 {"imageID": 2}
2023-08-15T22:56:06.790 DEBUG:teuthology.provision.fog.smithi154:Requesting image smithi_ubuntu_20.04 (ID 2)
2023-08-15T22:56:06.790 DEBUG:teuthology.provision.fog.smithi154:FOG request: PUT /host/259 {"imageID": 2}
2023-08-15T22:56:11.794 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of ubuntu 20.04
2023-08-15T22:56:11.794 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None
2023-08-15T22:56:16.841 INFO:teuthology.provision.fog.smithi154:Scheduling deploy of ubuntu 20.04
2023-08-15T22:56:16.841 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None
2023-08-15T22:56:21.849 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /tasktype {"name": "deploy"}
2023-08-15T22:56:25.008 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /tasktype {"name": "deploy"}
2023-08-15T22:56:25.010 DEBUG:teuthology.provision.fog.smithi002:FOG request: POST /host/108/task {"taskTypeID": 1}
2023-08-15T22:56:25.042 DEBUG:teuthology.provision.fog.smithi154:FOG request: POST /host/259/task {"taskTypeID": 1}
2023-08-15T22:56:25.061 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None
2023-08-15T22:56:29.997 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None
2023-08-15T22:56:30.060 INFO:teuthology.orchestra.console.smithi002:Power off
2023-08-15T22:56:30.061 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-15T22:56:30.083 DEBUG:teuthology.orchestra.console.smithi002:power off output: Chassis Power Control: Down/Off
2023-08-15T22:56:30.083 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:56:30.105 INFO:teuthology.orchestra.console.smithi154:Power off
2023-08-15T22:56:30.106 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-15T22:56:30.119 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off
2023-08-15T22:56:30.142 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off
2023-08-15T22:56:30.143 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:56:30.177 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on
2023-08-15T22:56:30.221 INFO:teuthology.orchestra.console.smithi002:Power off completed
2023-08-15T22:56:30.322 INFO:teuthology.orchestra.console.smithi002:Power on
2023-08-15T22:56:30.322 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-08-15T22:56:30.345 DEBUG:teuthology.orchestra.console.smithi002:power on output: Chassis Power Control: Up/On
2023-08-15T22:56:30.345 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:56:30.366 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off
2023-08-15T22:56:34.180 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:56:34.291 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on
2023-08-15T22:56:34.367 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:56:34.477 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on
2023-08-15T22:56:34.578 INFO:teuthology.orchestra.console.smithi002:Power on completed
2023-08-15T22:56:34.678 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish
2023-08-15T22:56:34.679 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None
2023-08-15T22:56:39.684 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:56:39.795 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off
2023-08-15T22:56:39.895 INFO:teuthology.orchestra.console.smithi154:Power off completed
2023-08-15T22:56:39.996 INFO:teuthology.orchestra.console.smithi154:Power on
2023-08-15T22:56:39.997 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-08-15T22:56:40.018 DEBUG:teuthology.orchestra.console.smithi154:power on output: Chassis Power Control: Up/On
2023-08-15T22:56:40.018 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:56:40.038 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off
2023-08-15T22:56:44.040 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:56:44.149 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off
2023-08-15T22:56:48.152 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:56:49.261 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:56:53.263 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:56:54.372 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:56:54.746 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None
2023-08-15T22:56:59.752 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:57:00.864 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:57:04.868 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:57:05.977 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:57:09.979 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:57:11.088 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:57:14.832 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None
2023-08-15T22:57:19.841 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:57:20.950 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:57:24.955 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:57:26.065 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:57:30.069 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:57:31.179 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:57:34.901 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None
2023-08-15T22:57:39.909 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:57:41.020 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:57:45.022 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:57:46.131 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:57:50.136 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:57:51.247 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:57:54.977 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None
2023-08-15T22:57:59.992 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:58:01.104 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:58:05.108 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:58:06.218 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:58:06.218 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-08-15T22:58:06.219 DEBUG:teuthology.provision.fog.smithi154:Canceling deploy task with ID None
2023-08-15T22:58:06.219 DEBUG:teuthology.provision.fog.smithi154:FOG request: DELETE /task/cancel {"id": None}
2023-08-15T22:58:07.533 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi154
2023-08-15T22:58:07.534 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-15T22:58:12.683 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=68c1deefdc454234b31103d3948dd23f
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi154
2023-08-15T22:58:12.685 INFO:teuthology.nuke:Checking targets against current locks
2023-08-15T22:58:22.697 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None
2023-08-15T22:58:33.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-15T22:58:33.045 DEBUG:teuthology.nuke:shortname: smithi002
2023-08-15T22:58:33.045 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-08-15T22:58:33.069 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-08-15_16:56:50-fs-wip-yuri7-testing-2023-08-15-0741-pacific-distro-default-smithi/7368962', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-15 22:55:20.972814', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH2C3dd/4taZgZ+Oldi/LDg1eWsL2xHQfhXCtwRdft7c/vC1zB5MVEU8x52MLWokGHo2OI248rB6i++J4oDgQRE='}
2023-08-15T22:58:33.071 DEBUG:teuthology.nuke:shortname: smithi154
2023-08-15T22:58:33.071 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-08-15T22:58:33.085 ERROR:teuthology.orchestra.connection:Error authenticating with smithi002.front.sepia.ceph.com: Authentication failed.
2023-08-15T22:58:33.088 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-08-15T22:58:33.089 DEBUG:teuthology.orchestra.run.smithi002:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-08-15T22:58:33.091 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-15T22:58:33.094 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-08-15_16:56:50-fs-wip-yuri7-testing-2023-08-15-0741-pacific-distro-default-smithi/7368962', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-15 22:55:20.974144', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAqyVivUHpkiaP+S0YV7UudskFdy3QdaAny2n/kBr/tm2XRK+En1U1cdwZ32A2PEZ9Wu1IuvMUjWYDp2BYKxCcA='}
2023-08-15T22:58:33.126 INFO:teuthology.orchestra.console.smithi002:Power off
2023-08-15T22:58:33.127 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-15T22:58:33.151 DEBUG:teuthology.orchestra.console.smithi002:power off output: Chassis Power Control: Down/Off
2023-08-15T22:58:33.152 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:58:33.171 INFO:teuthology.orchestra.console.smithi154:Power off
2023-08-15T22:58:33.172 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-15T22:58:33.183 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on
2023-08-15T22:58:33.196 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off
2023-08-15T22:58:33.196 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:58:33.222 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on
2023-08-15T22:58:34.098 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-15T22:58:35.103 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-15T22:58:36.108 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-15T22:58:37.113 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-15T22:58:37.185 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:58:37.224 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:58:38.120 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-15T22:58:38.296 DEBUG:teuthology.orchestra.console.smithi002:check power output: 
2023-08-15T22:58:38.333 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:58:39.127 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-15T22:58:40.133 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-15T22:58:41.138 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-15T22:58:42.297 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:58:42.334 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:58:43.409 DEBUG:teuthology.orchestra.console.smithi002:check power output: 
2023-08-15T22:58:43.445 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:58:47.411 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:58:47.446 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:58:48.522 DEBUG:teuthology.orchestra.console.smithi002:check power output: 
2023-08-15T22:58:48.556 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:58:52.523 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:58:52.557 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:58:53.634 DEBUG:teuthology.orchestra.console.smithi002:check power output: 
2023-08-15T22:58:53.666 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:58:57.635 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:58:57.667 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:58:58.746 DEBUG:teuthology.orchestra.console.smithi002:check power output: 
2023-08-15T22:58:58.776 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:59:02.748 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:59:02.777 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:59:03.857 DEBUG:teuthology.orchestra.console.smithi002:check power output: 
2023-08-15T22:59:03.887 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:59:07.859 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:59:07.888 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:59:08.968 DEBUG:teuthology.orchestra.console.smithi002:check power output: 
2023-08-15T22:59:08.997 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:59:12.970 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:59:12.998 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:59:14.082 DEBUG:teuthology.orchestra.console.smithi002:check power output: 
2023-08-15T22:59:14.110 DEBUG:teuthology.orchestra.console.smithi154:check power output: 
2023-08-15T22:59:18.084 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:59:18.110 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-15T22:59:18.220 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off
2023-08-15T22:59:18.321 INFO:teuthology.orchestra.console.smithi154:Power off completed
2023-08-15T22:59:23.432 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off
2023-08-15T22:59:28.457 INFO:teuthology.orchestra.console.smithi002:Power off completed
2023-08-15T22:59:28.487 INFO:teuthology.lock.ops:unlocked: smithi154.front.sepia.ceph.com
2023-08-15T22:59:38.620 INFO:teuthology.lock.ops:unlocked: smithi002.front.sepia.ceph.com
2023-08-15T22:59:38.849 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi154