2022-04-20T20:54:16.869 INFO:root:teuthology version: 1.1.0-b37d6865
2022-04-20T20:54:16.993 INFO:teuthology.lock.ops:Start node 'smithi070.front.sepia.ceph.com' reimaging
2022-04-20T20:54:16.994 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: reset os type and version on server
2022-04-20T20:54:16.994 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server
2022-04-20T20:54:17.016 INFO:teuthology.lock.ops:Node 'smithi070.front.sepia.ceph.com' reimaging is complete
2022-04-20T20:54:17.016 INFO:teuthology.lock.ops:Start node 'smithi146.front.sepia.ceph.com' reimaging
2022-04-20T20:54:17.016 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: reset os type and version on server
2022-04-20T20:54:17.016 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2022-04-20T20:54:17.037 INFO:teuthology.lock.ops:Node 'smithi146.front.sepia.ceph.com' reimaging is complete
2022-04-20T20:54:17.225 INFO:teuthology.provision.fog.smithi070:Scheduling deploy of rhel 8.4
2022-04-20T20:54:17.241 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of rhel 8.4
2022-04-20T20:54:17.550 INFO:teuthology.orchestra.console:Power off smithi146
2022-04-20T20:54:17.563 INFO:teuthology.orchestra.console:Power off smithi070
2022-04-20T20:54:17.695 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2022-04-20T20:54:17.706 INFO:teuthology.orchestra.console:Power off for smithi070 completed
2022-04-20T20:54:17.797 INFO:teuthology.orchestra.console:Power on smithi146
2022-04-20T20:54:17.808 INFO:teuthology.orchestra.console:Power on smithi070
2022-04-20T20:54:26.165 INFO:teuthology.orchestra.console:Power on for smithi146 completed
2022-04-20T20:54:26.170 INFO:teuthology.orchestra.console:Power on for smithi070 completed
2022-04-20T20:54:26.267 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish
2022-04-20T20:54:26.272 INFO:teuthology.provision.fog.smithi070:Waiting for deploy to finish
2022-04-20T20:58:59.308 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-20T20:58:59.525 INFO:teuthology.provision.fog.smithi070:Node is ready
2022-04-20T20:58:59.539 INFO:teuthology.orchestra.run.smithi070.stdout:smithi070.front.sepia.ceph.com
2022-04-20T20:58:59.594 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70 smithi070.front.sepia.ceph.com smithi070
2022-04-20T20:58:59.903 INFO:teuthology.provision.fog.smithi070:Deploy complete!
2022-04-20T20:59:15.956 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-20T20:59:16.164 INFO:teuthology.provision.fog.smithi146:Node is ready
2022-04-20T20:59:16.179 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com
2022-04-20T20:59:16.234 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146
2022-04-20T20:59:16.550 INFO:teuthology.provision.fog.smithi146:Deploy complete!
2022-04-20T20:59:16.694 INFO:teuthology.lock.ops:Checking smithi070.front.sepia.ceph.com
2022-04-20T20:59:16.694 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-20T20:59:16.720 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com
2022-04-20T20:59:16.720 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-20T20:59:16.736 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: set os type and version on server
2022-04-20T20:59:16.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com'
2022-04-20T20:59:16.926 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64
2022-04-20T20:59:16.981 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="Red Hat Enterprise Linux"
2022-04-20T20:59:16.981 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="8.4 (Ootpa)"
2022-04-20T20:59:16.981 INFO:teuthology.orchestra.run.smithi070.stdout:ID="rhel"
2022-04-20T20:59:16.982 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="fedora"
2022-04-20T20:59:16.982 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="8.4"
2022-04-20T20:59:16.982 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el8"
2022-04-20T20:59:16.982 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-04-20T20:59:16.982 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31"
2022-04-20T20:59:16.982 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-04-20T20:59:16.982 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://www.redhat.com/"
2022-04-20T20:59:16.982 INFO:teuthology.orchestra.run.smithi070.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-04-20T20:59:16.983 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-20T20:59:16.983 INFO:teuthology.orchestra.run.smithi070.stdout:
2022-04-20T20:59:16.983 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-20T20:59:16.983 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-04-20T20:59:16.983 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-04-20T20:59:16.983 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-04-20T20:59:16.984 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server
2022-04-20T20:59:17.002 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server
2022-04-20T20:59:17.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2022-04-20T20:59:17.221 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64
2022-04-20T20:59:17.275 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="Red Hat Enterprise Linux"
2022-04-20T20:59:17.275 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8.4 (Ootpa)"
2022-04-20T20:59:17.275 INFO:teuthology.orchestra.run.smithi146.stdout:ID="rhel"
2022-04-20T20:59:17.276 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="fedora"
2022-04-20T20:59:17.276 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8.4"
2022-04-20T20:59:17.276 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8"
2022-04-20T20:59:17.276 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2022-04-20T20:59:17.276 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31"
2022-04-20T20:59:17.276 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2022-04-20T20:59:17.276 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://www.redhat.com/"
2022-04-20T20:59:17.276 INFO:teuthology.orchestra.run.smithi146.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2022-04-20T20:59:17.276 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-20T20:59:17.276 INFO:teuthology.orchestra.run.smithi146.stdout:
2022-04-20T20:59:17.277 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-20T20:59:17.277 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2022-04-20T20:59:17.277 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2022-04-20T20:59:17.277 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2022-04-20T20:59:17.277 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2022-04-20T20:59:17.364 INFO:teuthology.dispatcher.supervisor:Running job 6798221
2022-04-20T20:59:17.364 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b37d686568ea3c673197ca892d3fb267463611f8/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-04-20_20:51:56-rados-wip-yuri4-testing-2022-04-18-1425-pacific-distro-default-smithi/6798221 --name yuriw-2022-04-20_20:51:56-rados-wip-yuri4-testing-2022-04-18-1425-pacific-distro-default-smithi --description rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/sync workloads/rados_mon_osdmap_prune} -- /home/teuthworker/archive/yuriw-2022-04-20_20:51:56-rados-wip-yuri4-testing-2022-04-18-1425-pacific-distro-default-smithi/6798221/orig.config.yaml
2022-04-20T20:59:17.368 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-04-20_20:51:56-rados-wip-yuri4-testing-2022-04-18-1425-pacific-distro-default-smithi/6798221
2022-04-20T20:59:17.369 INFO:teuthology.dispatcher.supervisor:Job PID: 28398
2022-04-20T20:59:17.369 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-04-20T21:01:17.427 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-04-20T21:01:17.528 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2022-04-20T21:01:17.528 INFO:teuthology.nuke:Checking targets against current locks
2022-04-20T21:01:17.660 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-20T21:01:17.663 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-20T21:01:17.737 INFO:teuthology.orchestra.console:Power off smithi146
2022-04-20T21:01:17.749 INFO:teuthology.orchestra.console:Power off smithi070
2022-04-20T21:01:21.994 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2022-04-20T21:01:22.000 INFO:teuthology.orchestra.console:Power off for smithi070 completed
2022-04-20T21:01:49.288 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b37d686568ea3c673197ca892d3fb267463611f8/virtualenv/bin/teuthology-dispatcher", line 33, in <module>
    sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')())
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b37d686568ea3c673197ca892d3fb267463611f8/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b37d686568ea3c673197ca892d3fb267463611f8/teuthology/dispatcher/__init__.py", line 61, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b37d686568ea3c673197ca892d3fb267463611f8/teuthology/dispatcher/supervisor.py", line 61, in main
    verbose
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b37d686568ea3c673197ca892d3fb267463611f8/teuthology/dispatcher/supervisor.py", line 154, in run_job
    unlock_targets(job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b37d686568ea3c673197ca892d3fb267463611f8/teuthology/dispatcher/supervisor.py", line 253, in unlock_targets
    nuke(fake_ctx, True)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b37d686568ea3c673197ca892d3fb267463611f8/teuthology/nuke/__init__.py", line 273, in nuke
    for unnuked in p:
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b37d686568ea3c673197ca892d3fb267463611f8/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b37d686568ea3c673197ca892d3fb267463611f8/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b37d686568ea3c673197ca892d3fb267463611f8/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b37d686568ea3c673197ca892d3fb267463611f8/teuthology/nuke/__init__.py", line 304, in nuke_one
    unlock_one(ctx, list(target.keys())[0], ctx.owner)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b37d686568ea3c673197ca892d3fb267463611f8/teuthology/lock/ops.py", line 209, in unlock_one
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b37d686568ea3c673197ca892d3fb267463611f8/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'unlock smithi070.front.sepia.ceph.com' reached maximum tries (10) after waiting for 32.5 seconds