2023-07-11T07:43:16.595 INFO:root:teuthology version: 0.0.1.dev140+gb53f236 2023-07-11T07:43:16.597 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T07:43:16.598 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T07:43:16.761 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc93b20f8b0>, signals=[15]) 2023-07-11T07:43:16.762 INFO:teuthology.lock.ops:Start node 'smithi100.front.sepia.ceph.com' reimaging 2023-07-11T07:43:16.763 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: reset os type and version on server 2023-07-11T07:43:16.763 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2023-07-11T07:43:16.799 INFO:teuthology.lock.ops:Start node 'smithi112.front.sepia.ceph.com' reimaging 2023-07-11T07:43:16.800 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: reset os type and version on server 2023-07-11T07:43:16.800 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2023-07-11T07:43:16.801 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /host {"name": "smithi100"} 2023-07-11T07:43:16.832 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /host {"name": "smithi112"} 2023-07-11T07:43:16.884 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-07-11T07:43:16.910 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-07-11T07:43:16.927 DEBUG:teuthology.provision.fog.smithi100:Requesting image smithi_rhel_8.4 (ID 7) 2023-07-11T07:43:16.927 DEBUG:teuthology.provision.fog.smithi100:FOG request: PUT /host/205 {"imageID": 7} 2023-07-11T07:43:16.950 DEBUG:teuthology.provision.fog.smithi112:Requesting image smithi_rhel_8.4 (ID 7) 2023-07-11T07:43:16.950 DEBUG:teuthology.provision.fog.smithi112:FOG request: PUT /host/217 {"imageID": 7} 2023-07-11T07:43:21.957 INFO:teuthology.provision.fog.smithi100:Scheduling deploy of rhel 8.4 2023-07-11T07:43:21.958 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-07-11T07:43:27.007 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of rhel 8.4 2023-07-11T07:43:27.007 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-07-11T07:43:32.015 DEBUG:teuthology.provision.fog.smithi100:Canceling deploy task with ID 376713 2023-07-11T07:43:32.015 DEBUG:teuthology.provision.fog.smithi100:FOG request: DELETE /task/cancel {"id": 376713} 2023-07-11T07:43:32.062 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /tasktype {"name": "deploy"} 2023-07-11T07:43:32.069 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /tasktype {"name": "deploy"} 2023-07-11T07:43:32.096 DEBUG:teuthology.provision.fog.smithi100:FOG request: POST /host/205/task {"taskTypeID": 1} 2023-07-11T07:43:32.102 DEBUG:teuthology.provision.fog.smithi112:FOG request: POST /host/217/task {"taskTypeID": 1} 2023-07-11T07:43:32.146 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-07-11T07:43:32.158 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-07-11T07:43:42.727 INFO:teuthology.orchestra.console:Power off smithi112 2023-07-11T07:43:42.728 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-11T07:43:42.743 INFO:teuthology.orchestra.console:Power off smithi100 2023-07-11T07:43:42.744 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-11T07:43:42.753 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:43:42.874 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2023-07-11T07:43:42.975 INFO:teuthology.orchestra.console:Power on smithi112 2023-07-11T07:43:42.976 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-11T07:43:42.997 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:43:47.024 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:43:47.236 INFO:teuthology.orchestra.console:Power on for smithi112 completed 2023-07-11T07:43:47.337 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish 2023-07-11T07:43:47.337 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-07-11T07:43:47.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T07:43:50.448 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-07-11T07:43:56.452 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T07:44:01.458 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:44:04.532 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-07-11T07:44:06.469 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:44:10.537 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T07:44:11.580 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:44:13.616 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-07-11T07:44:16.692 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:44:16.903 INFO:teuthology.orchestra.console:Power off for smithi100 completed 2023-07-11T07:44:17.004 INFO:teuthology.orchestra.console:Power on smithi100 2023-07-11T07:44:17.005 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-11T07:44:17.028 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:44:19.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T07:44:21.052 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:44:22.676 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-07-11T07:44:26.164 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:44:28.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T07:44:33.685 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:44:36.752 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-07-11T07:44:38.798 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:44:42.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T07:44:43.909 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:44:45.808 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-07-11T07:44:49.020 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:44:51.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T07:44:56.819 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:45:01.929 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:45:07.042 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:45:12.152 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:45:15.376 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-07-11T07:45:17.264 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:45:21.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T07:45:22.376 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:45:24.388 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-07-11T07:45:27.489 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:45:30.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T07:45:35.398 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:45:35.474 ERROR:teuthology.orchestra.connection:Error authenticating with smithi112.front.sepia.ceph.com: Authentication failed. 2023-07-11T07:45:35.477 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-11T07:45:35.477 DEBUG:teuthology.orchestra.run.smithi112:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-11T07:45:35.478 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-11T07:45:36.484 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-11T07:45:36.508 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds 2023-07-11T07:45:36.508 DEBUG:teuthology.provision.fog.smithi100:Canceling deploy task with ID 376714 2023-07-11T07:45:36.509 DEBUG:teuthology.provision.fog.smithi100:FOG request: DELETE /task/cancel {"id": 376714} 2023-07-11T07:45:41.518 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-11T07:45:41.566 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/provision/fog.py", line 85, in create self.remote.console.power_on() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/console.py", line 243, in power_on raise RuntimeError(err_msg) RuntimeError: Failed to power on smithi100 2023-07-11T07:45:41.566 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T07:45:41.567 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T07:45:41.644 INFO:teuthology.nuke:Checking targets against current locks 2023-07-11T07:45:41.699 DEBUG:teuthology.nuke:shortname: smithi100 2023-07-11T07:45:41.700 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-11T07:45:41.726 DEBUG:teuthology.nuke:shortname: smithi112 2023-07-11T07:45:41.726 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-11T07:45:41.729 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2023-07-11_06:48:53-orch:cephadm-cv-bluestore-rdr-quincy-distro-default-smithi/7333441', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-11 07:43:15.737880', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqT//hXweS/r1uwvZRFGOHsLutnrVDUi40TrKXLEXgdK+JCWuctDSktQ4s/BYgRhDsyLZpMuDb3iEASTpusaL2M3Tw2PQ1VXN5nwirsyX0aTA2iyX/eKxr5otHFFHIb0Va5HGmq4cDIz4tK0WBguONh751HcMEJA4WWPv4hL2CRbSYb9Sq6o38bjr6PxH3aOa1PcKdNW/pxvr/L69v1hJzAJ0Jj1uUn9ojbEbTxGZI5cWMV3ZTicwf0treD/sas6eTEEmW9IiYq+0NIRT+TCiNBghmQrQC37EZDgwR8H3JC+wR0ne+GVtkyT1ctb1QLLfB5NjcIVa7yGohXBc8M+N6LXRceoSaXPC+FC/veFWZHjOoVk+8hbrk6rQw4TuZnFk1Ke8sX/xsQVRjC4YzQYDmc6XJduFbtqRzbeNkj9tcrGWNiP0j4XhvYyXgvJgyEVBocNC5gEk3QWnEJmKew/T3PUzznFV+lM2N62wxzCLmhkDrUMgG+10sj2dZfhAwHM='} 2023-07-11T07:45:41.753 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2023-07-11_06:48:53-orch:cephadm-cv-bluestore-rdr-quincy-distro-default-smithi/7333441', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-11 07:43:15.739439', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDvuN4D/05RLSZ9CSyJVg1vLUSXlRD0uAOCzdojKGEqxGmXPakClwsrNIUw1+ZrehoL1jyItms4Dc9NYheEYf/Buwj9LTrHewEX3Od4XdAeXMjWDP/Ecd279ZpVg9H1V1IOhJoFqRge7judnhbo5YVymvii+9ZaaJ0hx7Fm5HrETreOOv2Z8OKQOVi0tu2EIGR4HSJONCcCsJ52JDZ4DQ9jC3E3WU2xF5Ed1p3upLJcm7Uena5QY+0EsG96PivWAF5pKKiMtwe3Hp+iI82ES76OulNazGP5JpCMdbx1SV9fnmanOSJ5mJslro/NokhopjSoIzbp4H9fU/O8kTuYsftNISykpFVHzxLubhRPY566KwmyaK+jDXE8Wb7R2PEZflBN+CFIB7vEGPvuBIjLohiW6QPeW3S3XpcUyMwxPuGDclgLl19VK8Jq5Gal1RhsgukHUEje+EysboYzPkM4RlPZ0dcKa5/XT/bZuuPzNxXiq0HPzyHkIxQViJV6+jBKA0c='} 2023-07-11T07:45:41.784 INFO:teuthology.orchestra.console:Power off smithi100 2023-07-11T07:45:41.785 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-11T07:45:41.805 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:45:41.826 INFO:teuthology.orchestra.console:Power off smithi112 2023-07-11T07:45:41.827 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-11T07:45:41.851 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:45:42.525 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-11T07:45:43.530 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-11T07:45:44.535 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-11T07:45:45.540 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-11T07:45:45.838 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:45:45.874 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:45:46.551 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-11T07:45:47.558 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-11T07:45:48.563 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-11T07:45:49.569 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-11T07:45:50.948 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:45:50.983 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T07:45:51.192 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2023-07-11T07:46:01.337 INFO:teuthology.orchestra.console:Power off for smithi100 completed 2023-07-11T07:46:01.363 INFO:teuthology.lock.ops:unlocked: smithi112.front.sepia.ceph.com 2023-07-11T07:46:08.364 INFO:teuthology.lock.ops:unlocked: smithi100.front.sepia.ceph.com 2023-07-11T07:46:13.629 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/provision/fog.py", line 85, in create self.remote.console.power_on() File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/console.py", line 243, in power_on raise RuntimeError(err_msg) RuntimeError: Failed to power on smithi100