2023-06-26T23:38:47.403 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb
2023-06-26T23:38:47.404 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T23:38:47.405 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T23:38:51.915 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f56dfe57730>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f56e10748b0>, signals=[15])
2023-06-26T23:38:51.917 INFO:teuthology.lock.ops:Start node 'smithi038.front.sepia.ceph.com' reimaging
2023-06-26T23:38:51.917 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: reset os type and version on server
2023-06-26T23:38:51.918 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2023-06-26T23:38:51.941 INFO:teuthology.lock.ops:Start node 'smithi154.front.sepia.ceph.com' reimaging
2023-06-26T23:38:51.941 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: reset os type and version on server
2023-06-26T23:38:51.941 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server
2023-06-26T23:38:51.943 DEBUG:teuthology.provision.fog.smithi038:FOG request: GET /host {"name": "smithi038"}
2023-06-26T23:39:10.580 INFO:teuthology.lock.ops:Start node 'smithi179.front.sepia.ceph.com' reimaging
2023-06-26T23:39:10.580 INFO:teuthology.lock.ops:Updating [smithi179.front.sepia.ceph.com]: reset os type and version on server
2023-06-26T23:39:10.581 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server
2023-06-26T23:39:10.582 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /host {"name": "smithi154"}
2023-06-26T23:39:19.162 DEBUG:teuthology.provision.fog.smithi038:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-06-26T23:39:24.172 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /host {"name": "smithi179"}
2023-06-26T23:39:35.645 DEBUG:teuthology.provision.fog.smithi038:Requesting image smithi_centos_8.stream (ID 3)
2023-06-26T23:39:35.646 DEBUG:teuthology.provision.fog.smithi038:FOG request: PUT /host/143 {"imageID": 3}
2023-06-26T23:39:35.670 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-06-26T23:39:40.696 DEBUG:teuthology.provision.fog.smithi154:Requesting image smithi_centos_8.stream (ID 3)
2023-06-26T23:39:40.697 DEBUG:teuthology.provision.fog.smithi154:FOG request: PUT /host/259 {"imageID": 3}
2023-06-26T23:39:40.699 INFO:teuthology.provision.fog.smithi038:Scheduling deploy of centos 8.stream
2023-06-26T23:39:40.699 DEBUG:teuthology.provision.fog.smithi038:FOG request: GET /task/active None
2023-06-26T23:39:50.735 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-06-26T23:39:55.744 DEBUG:teuthology.provision.fog.smithi038:FOG request: GET /tasktype {"name": "deploy"}
2023-06-26T23:39:55.745 INFO:teuthology.provision.fog.smithi154:Scheduling deploy of centos 8.stream
2023-06-26T23:39:55.745 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None
2023-06-26T23:40:05.799 DEBUG:teuthology.provision.fog.smithi179:Requesting image smithi_centos_8.stream (ID 3)
2023-06-26T23:40:05.799 DEBUG:teuthology.provision.fog.smithi179:FOG request: PUT /host/284 {"imageID": 3}
2023-06-26T23:40:10.807 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /tasktype {"name": "deploy"}
2023-06-26T23:40:10.808 DEBUG:teuthology.provision.fog.smithi038:FOG request: POST /host/143/task {"taskTypeID": 1}
2023-06-26T23:40:15.861 DEBUG:teuthology.provision.fog.smithi154:FOG request: POST /host/259/task {"taskTypeID": 1}
2023-06-26T23:40:15.866 INFO:teuthology.provision.fog.smithi179:Scheduling deploy of centos 8.stream
2023-06-26T23:40:15.866 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-06-26T23:40:15.869 DEBUG:teuthology.provision.fog.smithi038:FOG request: GET /task/active None
2023-06-26T23:40:15.907 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None
2023-06-26T23:40:15.927 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /tasktype {"name": "deploy"}
2023-06-26T23:40:15.960 INFO:teuthology.orchestra.console:Power off smithi038
2023-06-26T23:40:15.961 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-26T23:40:15.971 DEBUG:teuthology.provision.fog.smithi179:FOG request: POST /host/284/task {"taskTypeID": 1}
2023-06-26T23:40:15.983 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:16.010 INFO:teuthology.orchestra.console:Power off smithi154
2023-06-26T23:40:16.011 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-26T23:40:16.033 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-06-26T23:40:16.035 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:16.142 INFO:teuthology.orchestra.console:Power off smithi179
2023-06-26T23:40:16.143 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-26T23:40:16.165 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:20.024 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:20.059 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:20.189 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:25.137 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:25.171 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:25.301 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:25.383 INFO:teuthology.orchestra.console:Power off for smithi154 completed
2023-06-26T23:40:25.485 INFO:teuthology.orchestra.console:Power on smithi154
2023-06-26T23:40:25.486 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-26T23:40:30.251 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:30.411 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:30.515 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:35.362 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:35.525 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:35.534 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:35.574 INFO:teuthology.orchestra.console:Power off for smithi038 completed
2023-06-26T23:40:35.676 INFO:teuthology.orchestra.console:Power on smithi038
2023-06-26T23:40:35.676 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-26T23:40:35.737 INFO:teuthology.orchestra.console:Power on for smithi154 completed
2023-06-26T23:40:35.838 INFO:teuthology.provision.fog.smithi154:Waiting for deploy to finish
2023-06-26T23:40:35.839 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None
2023-06-26T23:40:40.646 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:40.699 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:44.724 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:45.758 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:49.835 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:50.870 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:40:50.907 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None
2023-06-26T23:40:59.914 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:41:00.917 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:41:01.129 INFO:teuthology.orchestra.console:Power off for smithi179 completed
2023-06-26T23:41:01.230 INFO:teuthology.orchestra.console:Power on smithi179
2023-06-26T23:41:01.230 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-26T23:41:01.253 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:41:05.027 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:41:05.237 INFO:teuthology.orchestra.console:Power on for smithi038 completed
2023-06-26T23:41:05.277 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:41:05.339 INFO:teuthology.provision.fog.smithi038:Waiting for deploy to finish
2023-06-26T23:41:05.340 DEBUG:teuthology.provision.fog.smithi038:FOG request: GET /task/active None
2023-06-26T23:41:09.388 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:41:10.982 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None
2023-06-26T23:41:14.499 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:41:19.612 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:41:20.411 DEBUG:teuthology.provision.fog.smithi038:FOG request: GET /task/active None
2023-06-26T23:41:26.564 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None
2023-06-26T23:41:31.569 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:41:36.680 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:41:40.480 DEBUG:teuthology.provision.fog.smithi038:FOG request: GET /task/active None
2023-06-26T23:41:40.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T23:41:41.792 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:41:43.952 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38
2023-06-26T23:41:46.635 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None
2023-06-26T23:41:46.754 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T23:41:46.903 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:41:49.808 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-26T23:41:49.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T23:41:54.961 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:41:55.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T23:42:00.072 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:42:05.185 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:42:10.297 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:42:14.132 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-26T23:42:15.407 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:42:20.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T23:42:25.143 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:42:26.252 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-06-26T23:42:26.253 DEBUG:teuthology.provision.fog.smithi179:Canceling deploy task with ID 346325
2023-06-26T23:42:26.253 DEBUG:teuthology.provision.fog.smithi179:FOG request: DELETE /task/cancel {"id": 346325}
2023-06-26T23:42:27.107 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/dispatcher/supervisor.py", line 227, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/console.py", line 243, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi179
2023-06-26T23:42:27.108 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T23:42:27.109 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T23:42:29.581 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-26T23:42:29.582 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-26T23:42:29.583 DEBUG:teuthology.orchestra.run.smithi038:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-26T23:42:29.673 INFO:teuthology.nuke:Checking targets against current locks
2023-06-26T23:42:29.730 INFO:teuthology.provision.fog.smithi038:Node is ready
2023-06-26T23:42:29.730 DEBUG:teuthology.orchestra.run.smithi038:> hostname
2023-06-26T23:42:29.730 DEBUG:teuthology.nuke:shortname: smithi038
2023-06-26T23:42:29.731 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-26T23:42:34.754 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038.front.sepia.ceph.com
2023-06-26T23:42:34.755 DEBUG:teuthology.orchestra.run.smithi038:> grep smithi038.front.sepia.ceph.com /etc/hosts
2023-06-26T23:42:34.755 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-26_18:56:05-rados-wip-yuri7-testing-2023-06-23-1022-distro-default-smithi/7317128', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-26 23:38:30.084089', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdZfWnvQ0cQtWAFULrzNn4qzQsnsOddbtAMA5TOF1bYT5PS5PIip3Af6vQPtDWoMO/e6q2SqB3eom31L54btCtT8ot1keldOQOe2edIQLqPLgiInQPcI5grcqIjDEX85wO+ODWK2I0XOhMpfsJ3RfDpZtbS6AOXRYQhOKGVSbgVRNvViEW+8w6fK/4dYzaxqjRJpNwMtYkpRBQEnWReijDC3zV5voKVK+IrJDEdARWOhP3hP+D+cX+T2+U94SkLFL437bZqC5pJiG8aJHO4Fn8h+gJynnd6Yngcr8L1OKkfrQ16sdXnIhmsCsPwjZ0c8xnyx5lDj9GYpiqjudL9rgXmIl/5kEVNdXaeE8txefIJRduAWtKRjoV9jm/9upZ/CgWpdFB3GD715thvNMePAHmI5JiKXXi+9XGJYL/eB8syW4aUAJwyyKhfj59oEBoD8z5m97ULKrWnUtx0GoEiL2vJybAVAVADVQT/SSpGxqDuBjWmZzGODseRB8dWLhjJVc='}
2023-06-26T23:42:34.756 DEBUG:teuthology.nuke:shortname: smithi154
2023-06-26T23:42:34.757 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-26T23:42:44.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T23:42:49.777 DEBUG:teuthology.nuke:shortname: smithi179
2023-06-26T23:42:49.777 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-26T23:42:49.800 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38 smithi038.front.sepia.ceph.com smithi038
2023-06-26T23:42:49.800 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname smithi038
2023-06-26T23:42:49.808 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-26_18:56:05-rados-wip-yuri7-testing-2023-06-23-1022-distro-default-smithi/7317128', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-26 23:38:30.086474', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBMyI/KHOA1aoAC1+CXz8RC0HLKP8cRdAzbUORWx+HOJ+I00m+WTmL3yDEhbaxjvZC+BerNfPxy9dDHgd3mqEROvUuEu3wGVoPotiyQt3INZGWTgxEhLAodnhLfr5hoLO6DUYEul5Jx1uXBOl9V5IGRL/9osb/umcxDtqN5Lq/m7GhWwuY+cYMhx/xwvUZV1Wivr8hzonz7Ha3Z588AxGUSgLd4Ye+uy///uPG1sObIjYV4R6mWjBXv+7ZxV4yujQzffzLByxX0nO/DDCHqjtBOKUckzWq2Am8Qfu+qZBmSj/gcdo8YVUak6u3fmZ4MT5+XZtxXy+u5X694nJJoFGtnXg1oR3hLGLZV9Mvx0sqFSpZyjrc9ffiwg64m5teWyqpXd0IAHVuJli+MN7CHD4qWvEWP9QB/hwhf0pPXrs45uwHlpIDLuXJRkpnKL0cTtfHBI3T78L0mnnX78Wpr+o3isjOIpLjXDE28esC1dZo2/Nn0Eu63m4cIAmpfth0wgU='}
2023-06-26T23:42:49.809 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-26_18:56:05-rados-wip-yuri7-testing-2023-06-23-1022-distro-default-smithi/7317128', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-26 23:38:30.088493', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCnxi9EifABvR/Qfo2BwLCP0dg+RAhq7IEXkROFDEI+KOtSgF93EWsk/Pl2zKoO0cALtlBT35IvzUlySScL1kOEVYQIqhyghm28omC3VAcrbwYmVGsgGTiWhg44EPKBlV1mxUnKCpO1ZiBH6qwCFvqTbzhy8b+U/xYuBcKvXo5SYopYaMTXMx1LSUecrEkiZsou6dhRyIbEqN54Ei9ML9+NLRZRW99PAwPRDQVXMGt2MM43QIl1wf3z5hEYz5dYdS7tnS5FhRfFlwJbhyTqmUwy2Wq9sLI9mRVLYTS0/oBH8p/JqKqaNlp04Gb3jjCf8qnDX+qprcv62zMugTDKgUdcJxCZTSuf+kXNNz8ZaaGvdX7pqILV1pX6WXL0U71B82+YbCLXHeRleatsKcoNGcobOE1PLePI5o3ETYX7H9Q9FHmoVcXJg3Scr3C2L0mt+AFDrEsB/Q/MKqTCOGZK3tzHLiRSQGWX/F1h6hUxyD7vjk6VWnBK/JiAgs/5XuCy+H8='}
2023-06-26T23:42:49.842 INFO:teuthology.orchestra.console:Power off smithi038
2023-06-26T23:42:49.842 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-26T23:42:49.872 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:42:49.897 INFO:teuthology.orchestra.console:Power off smithi179
2023-06-26T23:42:49.905 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-26T23:42:49.925 INFO:teuthology.orchestra.console:Power off smithi154
2023-06-26T23:42:49.926 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-26T23:42:49.935 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:42:49.949 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:42:49.962 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/smithi038.front.sepia.ceph.com/smithi038/g' /etc/hosts
2023-06-26T23:42:49.996 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/172.21.15.38/172.21.15.38/g' /etc/hosts
2023-06-26T23:42:50.068 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/smithi038.front.sepia.ceph.com/smithi038/g' /etc/hostname
2023-06-26T23:42:50.100 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-26T23:42:50.101 DEBUG:teuthology.orchestra.run.smithi154:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-26T23:42:50.140 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname smithi038
2023-06-26T23:42:50.211 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release
2023-06-26T23:42:50.271 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream"
2023-06-26T23:42:50.271 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="8"
2023-06-26T23:42:50.271 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos"
2023-06-26T23:42:50.273 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora"
2023-06-26T23:42:50.274 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="8"
2023-06-26T23:42:50.274 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el8"
2023-06-26T23:42:50.274 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 8"
2023-06-26T23:42:50.277 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31"
2023-06-26T23:42:50.278 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-06-26T23:42:50.278 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/"
2023-06-26T23:42:50.278 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-26T23:42:50.278 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-26T23:42:50.278 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-06-26T23:42:50.279 INFO:teuthology.provision.fog.smithi038:Deploy complete!
2023-06-26T23:42:50.310 INFO:teuthology.provision.fog.smithi154:Node is ready
2023-06-26T23:42:50.311 DEBUG:teuthology.orchestra.run.smithi154:> hostname
2023-06-26T23:42:50.330 INFO:teuthology.orchestra.run.smithi154.stdout:smithi154.front.sepia.ceph.com
2023-06-26T23:42:50.333 DEBUG:teuthology.orchestra.run.smithi154:> grep smithi154.front.sepia.ceph.com /etc/hosts
2023-06-26T23:42:50.390 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154 smithi154.front.sepia.ceph.com smithi154
2023-06-26T23:42:50.391 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname smithi154
2023-06-26T23:42:50.533 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/smithi154.front.sepia.ceph.com/smithi154/g' /etc/hosts
2023-06-26T23:42:50.564 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/172.21.15.154/172.21.15.154/g' /etc/hosts
2023-06-26T23:42:50.641 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/smithi154.front.sepia.ceph.com/smithi154/g' /etc/hostname
2023-06-26T23:42:50.710 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname smithi154
2023-06-26T23:42:50.778 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release
2023-06-26T23:42:50.836 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream"
2023-06-26T23:42:50.836 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="8"
2023-06-26T23:42:50.837 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos"
2023-06-26T23:42:50.837 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora"
2023-06-26T23:42:50.837 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="8"
2023-06-26T23:42:50.837 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el8"
2023-06-26T23:42:50.837 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 8"
2023-06-26T23:42:50.837 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31"
2023-06-26T23:42:50.838 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-06-26T23:42:50.838 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/"
2023-06-26T23:42:50.838 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-26T23:42:50.838 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-26T23:42:50.838 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-06-26T23:42:50.838 INFO:teuthology.provision.fog.smithi154:Deploy complete!
2023-06-26T23:42:53.936 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:42:53.964 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:42:54.963 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:42:58.051 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:42:58.075 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:42:58.286 INFO:teuthology.orchestra.console:Power off for smithi179 completed
2023-06-26T23:43:03.394 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:43:03.405 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T23:43:05.489 INFO:teuthology.lock.ops:unlocked: smithi179.front.sepia.ceph.com
2023-06-26T23:43:05.557 INFO:teuthology.orchestra.console:Power off for smithi038 completed
2023-06-26T23:43:10.615 INFO:teuthology.lock.ops:unlocked: smithi038.front.sepia.ceph.com
2023-06-26T23:43:10.656 INFO:teuthology.orchestra.console:Power off for smithi154 completed
2023-06-26T23:43:10.805 INFO:teuthology.lock.ops:unlocked: smithi154.front.sepia.ceph.com
2023-06-26T23:43:11.152 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/dispatcher/supervisor.py", line 57, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/dispatcher/supervisor.py", line 227, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/console.py", line 243, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi179