2023-07-05T11:54:52.164 INFO:root:teuthology version: 0.0.1.dev140+gb53f236
2023-07-05T11:54:52.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-05T11:54:52.166 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-05T11:54:52.313 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7eff6d3685e0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7eff6e5898b0>, signals=[15])
2023-07-05T11:54:52.315 INFO:teuthology.lock.ops:Start node 'smithi023.front.sepia.ceph.com' reimaging
2023-07-05T11:54:52.315 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: reset os type and version on server
2023-07-05T11:54:52.315 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server
2023-07-05T11:54:52.337 INFO:teuthology.lock.ops:Start node 'smithi070.front.sepia.ceph.com' reimaging
2023-07-05T11:54:52.337 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: reset os type and version on server
2023-07-05T11:54:52.337 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server
2023-07-05T11:54:52.339 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /host {"name": "smithi023"}
2023-07-05T11:54:52.362 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /host {"name": "smithi070"}
2023-07-05T11:54:52.430 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /image {"name": "smithi_ubuntu_22.04"}
2023-07-05T11:54:52.434 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /image {"name": "smithi_ubuntu_22.04"}
2023-07-05T11:54:52.471 DEBUG:teuthology.provision.fog.smithi023:Requesting image smithi_ubuntu_22.04 (ID 6)
2023-07-05T11:54:52.471 DEBUG:teuthology.provision.fog.smithi023:FOG request: PUT /host/129 {"imageID": 6}
2023-07-05T11:54:52.480 DEBUG:teuthology.provision.fog.smithi070:Requesting image smithi_ubuntu_22.04 (ID 6)
2023-07-05T11:54:52.480 DEBUG:teuthology.provision.fog.smithi070:FOG request: PUT /host/175 {"imageID": 6}
2023-07-05T11:54:52.519 INFO:teuthology.provision.fog.smithi070:Scheduling deploy of ubuntu 22.04
2023-07-05T11:54:52.519 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:54:52.520 INFO:teuthology.provision.fog.smithi023:Scheduling deploy of ubuntu 22.04
2023-07-05T11:54:52.521 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:54:52.645 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /tasktype {"name": "deploy"}
2023-07-05T11:54:52.652 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /tasktype {"name": "deploy"}
2023-07-05T11:54:52.683 DEBUG:teuthology.provision.fog.smithi070:FOG request: POST /host/175/task {"taskTypeID": 1}
2023-07-05T11:54:52.686 DEBUG:teuthology.provision.fog.smithi023:FOG request: POST /host/129/task {"taskTypeID": 1}
2023-07-05T11:54:52.746 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:54:52.757 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:54:52.907 INFO:teuthology.orchestra.console:Power off smithi023
2023-07-05T11:54:52.908 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-05T11:54:52.923 INFO:teuthology.orchestra.console:Power off smithi070
2023-07-05T11:54:52.924 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-05T11:54:52.934 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-05T11:54:52.946 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-05T11:54:56.961 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-05T11:54:56.973 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-05T11:55:01.076 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-05T11:55:01.083 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-05T11:55:01.286 INFO:teuthology.orchestra.console:Power off for smithi023 completed
2023-07-05T11:55:01.294 INFO:teuthology.orchestra.console:Power off for smithi070 completed
2023-07-05T11:55:01.388 INFO:teuthology.orchestra.console:Power on smithi023
2023-07-05T11:55:01.388 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-05T11:55:01.399 INFO:teuthology.orchestra.console:Power on smithi070
2023-07-05T11:55:01.400 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-05T11:55:01.411 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-05T11:55:01.421 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-05T11:55:05.438 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-05T11:55:05.446 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-05T11:55:09.553 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-05T11:55:09.561 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-05T11:55:09.764 INFO:teuthology.orchestra.console:Power on for smithi023 completed
2023-07-05T11:55:09.771 INFO:teuthology.orchestra.console:Power on for smithi070 completed
2023-07-05T11:55:09.865 INFO:teuthology.provision.fog.smithi023:Waiting for deploy to finish
2023-07-05T11:55:09.865 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:55:09.873 INFO:teuthology.provision.fog.smithi070:Waiting for deploy to finish
2023-07-05T11:55:09.873 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:55:25.026 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:55:25.027 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:55:40.206 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:55:40.208 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:55:55.384 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:55:55.385 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:56:10.569 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:56:10.570 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:56:25.761 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:56:25.763 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:56:40.990 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:56:40.992 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:56:56.180 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:56:56.181 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:57:11.412 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:57:11.413 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:57:26.628 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:57:26.630 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:57:41.869 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:57:41.871 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:57:57.103 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:57:57.105 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:58:12.330 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:58:12.332 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:58:27.529 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:58:27.530 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:58:42.743 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:58:42.744 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:58:57.951 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:58:57.952 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:59:13.148 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:59:13.149 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:59:28.326 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:59:28.327 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:59:43.500 DEBUG:teuthology.provision.fog.smithi023:FOG request: GET /task/active None
2023-07-05T11:59:43.501 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T11:59:43.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T11:59:46.704 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T11:59:52.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T11:59:55.792 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T11:59:58.660 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T12:00:01.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:00:04.848 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:00:10.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:00:13.836 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T12:00:13.904 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:00:19.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:00:22.996 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:00:29.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:00:29.001 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T12:00:32.080 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:00:38.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:00:41.172 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:00:44.154 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T12:00:47.176 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:00:50.256 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:00:56.260 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:00:59.284 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T12:00:59.316 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:01:05.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:01:08.400 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:01:14.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:01:14.417 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T12:01:17.456 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:01:23.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:01:26.512 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:01:29.532 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T12:01:32.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:01:35.568 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:01:41.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:01:44.639 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T12:01:44.656 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:01:50.660 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:01:53.712 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:01:59.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:01:59.748 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T12:02:02.800 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:02:08.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:02:11.888 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:02:14.856 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T12:02:17.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:02:20.948 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:02:26.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:02:29.956 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-05T12:02:30.032 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:02:30.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:02:33.104 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-07-05T12:02:36.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:02:39.092 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:02:39.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:02:42.160 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-07-05T12:02:45.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:02:48.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:02:48.176 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:02:51.216 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-07-05T12:02:54.180 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:02:57.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:02:57.232 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-07-05T12:03:00.272 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-07-05T12:03:03.236 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:03:06.276 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:03:09.328 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-07-05T12:03:15.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:03:18.384 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-07-05T12:03:24.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:04:03.277 WARNING:teuthology.provision.fog.smithi023:timed out
2023-07-05T12:04:09.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:04:24.406 WARNING:teuthology.provision.fog.smithi070:timed out
2023-07-05T12:04:24.784 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-05T12:04:24.785 DEBUG:teuthology.orchestra.run.smithi023:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-05T12:04:25.219 INFO:teuthology.provision.fog.smithi023:Node is ready
2023-07-05T12:04:25.220 DEBUG:teuthology.orchestra.run.smithi023:> hostname
2023-07-05T12:04:25.264 INFO:teuthology.orchestra.run.smithi023.stdout:smithi023.front.sepia.ceph.com
2023-07-05T12:04:25.265 DEBUG:teuthology.orchestra.run.smithi023:> grep smithi023.front.sepia.ceph.com /etc/hosts
2023-07-05T12:04:25.315 INFO:teuthology.orchestra.run.smithi023.stdout:172.21.15.23 smithi023.front.sepia.ceph.com smithi023
2023-07-05T12:04:25.315 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023
2023-07-05T12:04:25.490 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hosts
2023-07-05T12:04:25.505 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/172.21.15.23/172.21.15.23/g' /etc/hosts
2023-07-05T12:04:25.561 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hostname
2023-07-05T12:04:25.621 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023
2023-07-05T12:04:25.681 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release
2023-07-05T12:04:25.733 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-07-05T12:04:25.734 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="Ubuntu"
2023-07-05T12:04:25.734 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="22.04"
2023-07-05T12:04:25.734 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-07-05T12:04:25.734 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_CODENAME=jammy
2023-07-05T12:04:25.734 INFO:teuthology.orchestra.run.smithi023.stdout:ID=ubuntu
2023-07-05T12:04:25.735 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE=debian
2023-07-05T12:04:25.735 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://www.ubuntu.com/"
2023-07-05T12:04:25.735 INFO:teuthology.orchestra.run.smithi023.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-07-05T12:04:25.735 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-07-05T12:04:25.736 INFO:teuthology.orchestra.run.smithi023.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-07-05T12:04:25.736 INFO:teuthology.orchestra.run.smithi023.stdout:UBUNTU_CODENAME=jammy
2023-07-05T12:04:25.737 INFO:teuthology.provision.fog.smithi023:Deploy complete!
2023-07-05T12:04:25.739 DEBUG:teuthology.parallel:result is None
2023-07-05T12:04:30.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:04:30.667 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-05T12:04:30.668 DEBUG:teuthology.orchestra.run.smithi070:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-05T12:04:31.147 INFO:teuthology.provision.fog.smithi070:Node is ready
2023-07-05T12:04:31.147 DEBUG:teuthology.orchestra.run.smithi070:> hostname
2023-07-05T12:04:31.195 INFO:teuthology.orchestra.run.smithi070.stdout:smithi070.front.sepia.ceph.com
2023-07-05T12:04:31.195 DEBUG:teuthology.orchestra.run.smithi070:> grep smithi070.front.sepia.ceph.com /etc/hosts
2023-07-05T12:04:31.246 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70 smithi070.front.sepia.ceph.com smithi070
2023-07-05T12:04:31.246 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname smithi070
2023-07-05T12:04:31.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i -e 's/smithi070.front.sepia.ceph.com/smithi070/g' /etc/hosts
2023-07-05T12:04:31.415 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i -e 's/172.21.15.70/172.21.15.70/g' /etc/hosts
2023-07-05T12:04:31.473 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i -e 's/smithi070.front.sepia.ceph.com/smithi070/g' /etc/hostname
2023-07-05T12:04:31.534 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname smithi070
2023-07-05T12:04:31.591 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release
2023-07-05T12:04:31.638 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-07-05T12:04:31.638 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="Ubuntu"
2023-07-05T12:04:31.639 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="22.04"
2023-07-05T12:04:31.639 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-07-05T12:04:31.639 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_CODENAME=jammy
2023-07-05T12:04:31.639 INFO:teuthology.orchestra.run.smithi070.stdout:ID=ubuntu
2023-07-05T12:04:31.639 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE=debian
2023-07-05T12:04:31.639 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://www.ubuntu.com/"
2023-07-05T12:04:31.639 INFO:teuthology.orchestra.run.smithi070.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-07-05T12:04:31.640 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-07-05T12:04:31.640 INFO:teuthology.orchestra.run.smithi070.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-07-05T12:04:31.640 INFO:teuthology.orchestra.run.smithi070.stdout:UBUNTU_CODENAME=jammy
2023-07-05T12:04:31.640 INFO:teuthology.provision.fog.smithi070:Deploy complete!
2023-07-05T12:04:31.641 DEBUG:teuthology.parallel:result is None
2023-07-05T12:04:31.812 INFO:teuthology.lock.ops:Checking smithi023.front.sepia.ceph.com
2023-07-05T12:04:31.813 INFO:teuthology.lock.ops:New key found. Updating...
2023-07-05T12:04:31.837 INFO:teuthology.lock.ops:Checking smithi070.front.sepia.ceph.com
2023-07-05T12:04:31.837 INFO:teuthology.lock.ops:New key found. Updating...
2023-07-05T12:04:31.863 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: set os type and version on server
2023-07-05T12:04:31.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com'
2023-07-05T12:04:31.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:04:31.940 DEBUG:teuthology.orchestra.run.smithi023:> true
2023-07-05T12:04:32.041 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi023.front.sepia.ceph.com'
2023-07-05T12:04:32.042 DEBUG:teuthology.orchestra.run.smithi023:> uname -m
2023-07-05T12:04:32.089 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64
2023-07-05T12:04:32.089 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release
2023-07-05T12:04:32.137 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-07-05T12:04:32.137 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="Ubuntu"
2023-07-05T12:04:32.138 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="22.04"
2023-07-05T12:04:32.138 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-07-05T12:04:32.138 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_CODENAME=jammy
2023-07-05T12:04:32.139 INFO:teuthology.orchestra.run.smithi023.stdout:ID=ubuntu
2023-07-05T12:04:32.139 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE=debian
2023-07-05T12:04:32.139 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://www.ubuntu.com/"
2023-07-05T12:04:32.139 INFO:teuthology.orchestra.run.smithi023.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-07-05T12:04:32.139 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-07-05T12:04:32.140 INFO:teuthology.orchestra.run.smithi023.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-07-05T12:04:32.140 INFO:teuthology.orchestra.run.smithi023.stdout:UBUNTU_CODENAME=jammy
2023-07-05T12:04:32.140 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server
2023-07-05T12:04:32.162 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: set os type and version on server
2023-07-05T12:04:32.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com'
2023-07-05T12:04:32.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-05T12:04:32.238 DEBUG:teuthology.orchestra.run.smithi070:> true
2023-07-05T12:04:32.332 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com'
2023-07-05T12:04:32.333 DEBUG:teuthology.orchestra.run.smithi070:> uname -m
2023-07-05T12:04:32.378 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64
2023-07-05T12:04:32.378 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release
2023-07-05T12:04:32.426 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-07-05T12:04:32.426 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="Ubuntu"
2023-07-05T12:04:32.427 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="22.04"
2023-07-05T12:04:32.427 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-07-05T12:04:32.427 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_CODENAME=jammy
2023-07-05T12:04:32.427 INFO:teuthology.orchestra.run.smithi070.stdout:ID=ubuntu
2023-07-05T12:04:32.427 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE=debian
2023-07-05T12:04:32.428 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://www.ubuntu.com/"
2023-07-05T12:04:32.428 INFO:teuthology.orchestra.run.smithi070.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-07-05T12:04:32.428 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-07-05T12:04:32.428 INFO:teuthology.orchestra.run.smithi070.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-07-05T12:04:32.429 INFO:teuthology.orchestra.run.smithi070.stdout:UBUNTU_CODENAME=jammy
2023-07-05T12:04:32.429 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server
2023-07-05T12:04:32.470 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-05T12:04:32.471 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-05T12:04:32.566 INFO:teuthology.dispatcher.supervisor:Running job 7326964
2023-07-05T12:04:32.566 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2023-07-05_11:54:04-rados:mgr-wip-pdonnell-testing-20230705.003205-distro-default-smithi/7326964 --name pdonnell-2023-07-05_11:54:04-rados:mgr-wip-pdonnell-testing-20230705.003205-distro-default-smithi --description rados:mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/progress} -- /home/teuthworker/archive/pdonnell-2023-07-05_11:54:04-rados:mgr-wip-pdonnell-testing-20230705.003205-distro-default-smithi/7326964/orig.config.yaml
2023-07-05T12:04:32.570 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2023-07-05_11:54:04-rados:mgr-wip-pdonnell-testing-20230705.003205-distro-default-smithi/7326964
2023-07-05T12:04:32.570 INFO:teuthology.dispatcher.supervisor:Job PID: 16519
2023-07-05T12:04:32.571 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-07-05T12:06:32.669 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-05T12:06:32.670 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-05T12:08:32.854 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-05T12:08:32.854 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-05T12:10:33.030 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-05T12:10:33.032 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-05T12:12:33.209 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-05T12:12:33.209 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-05T12:14:33.381 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-05T12:14:33.382 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-05T12:16:33.545 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-05T12:16:33.547 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-05T12:18:33.707 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-05T12:18:33.708 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-05T12:18:38.783 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-07-05T12:18:52.799 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-07-05T12:18:52.800 INFO:teuthology.nuke:Checking targets against current locks
2023-07-05T12:18:52.848 DEBUG:teuthology.nuke:shortname: smithi023
2023-07-05T12:18:52.849 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-05T12:18:52.871 DEBUG:teuthology.nuke:shortname: smithi070
2023-07-05T12:18:52.873 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-05T12:18:52.875 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi023.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-05_11:54:04-rados:mgr-wip-pdonnell-testing-20230705.003205-distro-default-smithi/7326964', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-05 11:54:51.398776', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIamn1Ci5Fc+9v5F3ykYGnDtk7BQTNHqT8OalH1h/dLKtdzNTisvanIRVXmZJqFHKSJitnd8xkUste+QEiH/0NA='}
2023-07-05T12:18:52.900 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-05_11:54:04-rados:mgr-wip-pdonnell-testing-20230705.003205-distro-default-smithi/7326964', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-05 11:54:51.400527', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBBewe9p/HrJFFs3J6MfVhzJaCZqibbu7PD71EDAz55I'}
2023-07-05T12:18:52.929 INFO:teuthology.orchestra.console:Power off smithi023
2023-07-05T12:18:52.930 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-05T12:18:52.952 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-05T12:18:52.968 INFO:teuthology.orchestra.console:Power off smithi070
2023-07-05T12:18:52.969 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-05T12:18:52.990 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-05T12:18:53.079 INFO:teuthology.orchestra.console:Power off for smithi023 completed
2023-07-05T12:18:53.112 INFO:teuthology.orchestra.console:Power off for smithi070 completed
2023-07-05T12:18:53.230 INFO:teuthology.lock.ops:unlocked: smithi023.front.sepia.ceph.com
2023-07-05T12:18:53.264 INFO:teuthology.lock.ops:unlocked: smithi070.front.sepia.ceph.com