2023-02-22T10:07:29.177 INFO:root:teuthology version: 0.0.1.dev41+gfbbadb5
2023-02-22T10:07:29.290 INFO:teuthology.lock.ops:Start node 'smithi073.front.sepia.ceph.com' reimaging
2023-02-22T10:07:29.291 INFO:teuthology.lock.ops:Updating [smithi073.front.sepia.ceph.com]: reset os type and version on server
2023-02-22T10:07:29.291 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server
2023-02-22T10:07:29.307 INFO:teuthology.lock.ops:Node 'smithi073.front.sepia.ceph.com' reimaging is complete
2023-02-22T10:07:29.307 INFO:teuthology.lock.ops:Start node 'smithi087.front.sepia.ceph.com' reimaging
2023-02-22T10:07:29.308 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: reset os type and version on server
2023-02-22T10:07:29.308 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server
2023-02-22T10:07:29.323 INFO:teuthology.lock.ops:Node 'smithi087.front.sepia.ceph.com' reimaging is complete
2023-02-22T10:07:29.467 INFO:teuthology.provision.fog.smithi073:Scheduling deploy of centos 8.stream
2023-02-22T10:07:29.469 INFO:teuthology.provision.fog.smithi087:Scheduling deploy of centos 8.stream
2023-02-22T10:07:29.700 INFO:teuthology.orchestra.console:Power off smithi073
2023-02-22T10:07:29.734 INFO:teuthology.orchestra.console:Power off smithi087
2023-02-22T10:07:38.102 INFO:teuthology.orchestra.console:Power off for smithi087 completed
2023-02-22T10:07:38.203 INFO:teuthology.orchestra.console:Power on smithi087
2023-02-22T10:07:42.209 INFO:teuthology.orchestra.console:Power off for smithi073 completed
2023-02-22T10:07:42.310 INFO:teuthology.orchestra.console:Power on smithi073
2023-02-22T10:07:46.564 INFO:teuthology.orchestra.console:Power on for smithi073 completed
2023-02-22T10:07:46.571 INFO:teuthology.orchestra.console:Power on for smithi087 completed
2023-02-22T10:07:46.665 INFO:teuthology.provision.fog.smithi073:Waiting for deploy to finish
2023-02-22T10:07:46.672 INFO:teuthology.provision.fog.smithi087:Waiting for deploy to finish
2023-02-22T10:10:51.013 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.87
2023-02-22T10:10:51.013 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-02-22T10:11:00.069 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.87
2023-02-22T10:11:00.069 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-02-22T10:11:09.125 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.87
2023-02-22T10:11:09.259 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-02-22T10:11:18.341 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.87
2023-02-22T10:11:18.341 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-02-22T10:11:27.397 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.87
2023-02-22T10:11:27.397 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-02-22T10:12:33.461 WARNING:teuthology.provision.fog:timed out
2023-02-22T10:12:33.461 WARNING:teuthology.provision.fog:timed out
2023-02-22T10:12:39.747 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-02-22T10:12:39.765 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-02-22T10:12:39.978 INFO:teuthology.provision.fog.smithi087:Node is ready
2023-02-22T10:12:40.024 INFO:teuthology.provision.fog.smithi073:Node is ready
2023-02-22T10:12:40.035 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087.front.sepia.ceph.com
2023-02-22T10:12:40.082 INFO:teuthology.orchestra.run.smithi073.stdout:smithi073.front.sepia.ceph.com
2023-02-22T10:12:40.092 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87 smithi087.front.sepia.ceph.com smithi087
2023-02-22T10:12:40.140 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73 smithi073.front.sepia.ceph.com smithi073
2023-02-22T10:12:40.449 INFO:teuthology.provision.fog.smithi087:Deploy complete!
2023-02-22T10:12:40.499 INFO:teuthology.provision.fog.smithi073:Deploy complete!
2023-02-22T10:13:11.318 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/dispatcher/supervisor.py", line 209, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/lock/ops.py", line 285, in do_update_keys
    reference = query.list_locks(keyed_by_name=True)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/lock/query.py", line 67, in list_locks
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'list_locks' reached maximum tries (10) after waiting for 32.5 seconds
2023-02-22T10:13:11.375 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/yuriw-2023-02-20_23:16:20-rados-wip-yuri11-testing-2023-02-20-1329-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2023-02-22T10:13:12.436 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/yuriw-2023-02-20_23:16:20-rados-wip-yuri11-testing-2023-02-20-1329-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2023-02-22T10:13:14.401 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/yuriw-2023-02-20_23:16:20-rados-wip-yuri11-testing-2023-02-20-1329-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2023-02-22T10:13:17.276 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/yuriw-2023-02-20_23:16:20-rados-wip-yuri11-testing-2023-02-20-1329-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2023-02-22T10:13:21.064 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/yuriw-2023-02-20_23:16:20-rados-wip-yuri11-testing-2023-02-20-1329-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2023-02-22T10:13:25.759 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/yuriw-2023-02-20_23:16:20-rados-wip-yuri11-testing-2023-02-20-1329-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2023-02-22T10:13:31.363 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/yuriw-2023-02-20_23:16:20-rados-wip-yuri11-testing-2023-02-20-1329-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2023-02-22T10:13:37.877 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/yuriw-2023-02-20_23:16:20-rados-wip-yuri11-testing-2023-02-20-1329-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2023-02-22T10:13:45.303 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/yuriw-2023-02-20_23:16:20-rados-wip-yuri11-testing-2023-02-20-1329-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2023-02-22T10:13:53.780 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/yuriw-2023-02-20_23:16:20-rados-wip-yuri11-testing-2023-02-20-1329-distro-default-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2023-02-22T10:13:53.780 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/dispatcher/supervisor.py", line 209, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/lock/ops.py", line 285, in do_update_keys
    reference = query.list_locks(keyed_by_name=True)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/lock/query.py", line 67, in list_locks
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'list_locks' reached maximum tries (10) after waiting for 32.5 seconds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/dispatcher/__init__.py", line 63, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/dispatcher/supervisor.py", line 52, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/dispatcher/supervisor.py", line 213, in reimage
    report.try_push_job_info(ctx.config, dict(status='dead', failure_reason='Error reimaging machines: ' + str(e)))
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/report.py", line 508, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/report.py", line 473, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/report.py", line 298, in report_job
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_fbbadb5ff5cfccce0d20e136f8956e65ec955359/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'report job 7181674' reached maximum tries (10) after waiting for 50.89974201000152 seconds