2021-11-02T00:17:05.790 INFO:root:teuthology version: 1.1.0-c56135d1
2021-11-02T00:17:05.914 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi022.front.sepia.ceph.com' reimaging
2021-11-02T00:17:05.917 INFO:teuthology.lock.ops:Updating [ubuntu@smithi022.front.sepia.ceph.com]: reset os type and version on server
2021-11-02T00:17:05.917 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server
2021-11-02T00:17:05.935 INFO:teuthology.lock.ops:Node 'ubuntu@smithi022.front.sepia.ceph.com' reimaging is complete
2021-11-02T00:17:05.936 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi086.front.sepia.ceph.com' reimaging
2021-11-02T00:17:05.936 INFO:teuthology.lock.ops:Updating [ubuntu@smithi086.front.sepia.ceph.com]: reset os type and version on server
2021-11-02T00:17:05.937 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server
2021-11-02T00:17:05.958 INFO:teuthology.lock.ops:Node 'ubuntu@smithi086.front.sepia.ceph.com' reimaging is complete
2021-11-02T00:17:06.172 INFO:teuthology.provision.fog.smithi022:Scheduling deploy of ubuntu 20.04
2021-11-02T00:17:06.194 INFO:teuthology.provision.fog.smithi086:Scheduling deploy of ubuntu 20.04
2021-11-02T00:17:06.552 INFO:teuthology.orchestra.console:Power off smithi086
2021-11-02T00:17:06.571 INFO:teuthology.orchestra.console:Power off smithi022
2021-11-02T00:17:14.932 INFO:teuthology.orchestra.console:Power off for smithi022 completed
2021-11-02T00:17:15.111 INFO:teuthology.orchestra.console:Power on smithi022
2021-11-02T00:17:19.039 INFO:teuthology.orchestra.console:Power off for smithi086 completed
2021-11-02T00:17:19.140 INFO:teuthology.orchestra.console:Power on smithi086
2021-11-02T00:17:23.483 INFO:teuthology.orchestra.console:Power on for smithi022 completed
2021-11-02T00:17:23.584 INFO:teuthology.provision.fog.smithi022:Waiting for deploy to finish
2021-11-02T00:17:27.516 INFO:teuthology.orchestra.console:Power on for smithi086 completed
2021-11-02T00:17:27.617 INFO:teuthology.provision.fog.smithi086:Waiting for deploy to finish
2021-11-02T00:22:43.670 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-02T00:22:44.205 INFO:teuthology.provision.fog.smithi022:Node is ready
2021-11-02T00:22:44.212 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022.front.sepia.ceph.com
2021-11-02T00:22:44.219 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022
2021-11-02T00:22:44.391 INFO:teuthology.provision.fog.smithi022:Deploy complete!
2021-11-02T00:23:02.767 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-02T00:23:03.311 INFO:teuthology.provision.fog.smithi086:Node is ready
2021-11-02T00:23:03.318 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086.front.sepia.ceph.com
2021-11-02T00:23:03.367 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86 smithi086.front.sepia.ceph.com smithi086
2021-11-02T00:23:03.554 INFO:teuthology.provision.fog.smithi086:Deploy complete!
2021-11-02T00:23:03.804 INFO:teuthology.lock.ops:Checking smithi022.front.sepia.ceph.com
2021-11-02T00:23:03.808 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-02T00:23:03.829 INFO:teuthology.lock.ops:Checking smithi086.front.sepia.ceph.com
2021-11-02T00:23:03.830 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-02T00:23:03.849 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: set os type and version on server
2021-11-02T00:23:03.849 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com'
2021-11-02T00:23:04.111 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64
2021-11-02T00:23:04.118 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="Ubuntu"
2021-11-02T00:23:04.118 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-11-02T00:23:04.118 INFO:teuthology.orchestra.run.smithi022.stdout:ID=ubuntu
2021-11-02T00:23:04.119 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE=debian
2021-11-02T00:23:04.119 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-11-02T00:23:04.119 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="20.04"
2021-11-02T00:23:04.119 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://www.ubuntu.com/"
2021-11-02T00:23:04.119 INFO:teuthology.orchestra.run.smithi022.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-11-02T00:23:04.120 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-11-02T00:23:04.120 INFO:teuthology.orchestra.run.smithi022.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-11-02T00:23:04.120 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_CODENAME=focal
2021-11-02T00:23:04.120 INFO:teuthology.orchestra.run.smithi022.stdout:UBUNTU_CODENAME=focal
2021-11-02T00:23:04.122 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server
2021-11-02T00:23:04.142 INFO:teuthology.lock.ops:Updating [smithi086.front.sepia.ceph.com]: set os type and version on server
2021-11-02T00:23:04.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com'
2021-11-02T00:23:04.350 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64
2021-11-02T00:23:04.356 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Ubuntu"
2021-11-02T00:23:04.357 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-11-02T00:23:04.357 INFO:teuthology.orchestra.run.smithi086.stdout:ID=ubuntu
2021-11-02T00:23:04.357 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE=debian
2021-11-02T00:23:04.357 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-11-02T00:23:04.358 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="20.04"
2021-11-02T00:23:04.358 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.ubuntu.com/"
2021-11-02T00:23:04.358 INFO:teuthology.orchestra.run.smithi086.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-11-02T00:23:04.358 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-11-02T00:23:04.359 INFO:teuthology.orchestra.run.smithi086.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-11-02T00:23:04.359 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_CODENAME=focal
2021-11-02T00:23:04.359 INFO:teuthology.orchestra.run.smithi086.stdout:UBUNTU_CODENAME=focal
2021-11-02T00:23:04.360 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server
2021-11-02T00:23:04.451 INFO:teuthology.dispatcher.supervisor:Running job 6474028
2021-11-02T00:23:04.452 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/bin/teuthology -v --owner scheduled_nojha@teuthology --archive /home/teuthworker/archive/nojha-2021-11-01_23:58:17-upgrade:octopus-x-master-distro-basic-smithi/6474028 --name nojha-2021-11-01_23:58:17-upgrade:octopus-x-master-distro-basic-smithi --description upgrade:octopus-x/rgw-multisite/{clusters frontend overrides realm tasks upgrade/secondary} -- /home/teuthworker/archive/nojha-2021-11-01_23:58:17-upgrade:octopus-x-master-distro-basic-smithi/6474028/orig.config.yaml
2021-11-02T00:23:04.458 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nojha-2021-11-01_23:58:17-upgrade:octopus-x-master-distro-basic-smithi/6474028
2021-11-02T00:23:04.459 INFO:teuthology.dispatcher.supervisor:Job PID: 11861
2021-11-02T00:23:04.459 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-02T00:29:04.826 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-02T00:29:04.985 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-11-02T00:29:04.986 INFO:teuthology.nuke:Checking targets against current locks
2021-11-02T00:29:05.118 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-02T00:29:05.122 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-02T00:29:05.198 INFO:teuthology.orchestra.console:Power off smithi086
2021-11-02T00:29:05.215 INFO:teuthology.orchestra.console:Power off smithi022
2021-11-02T00:29:13.579 INFO:teuthology.orchestra.console:Power off for smithi022 completed
2021-11-02T00:29:13.729 INFO:teuthology.lock.ops:unlocked: smithi022.front.sepia.ceph.com
2021-11-02T00:29:17.687 INFO:teuthology.orchestra.console:Power off for smithi086 completed
2021-11-02T00:29:44.972 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/bin/teuthology-dispatcher", line 33, in <module>
    sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')())
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/scripts/dispatcher.py", line 33, in main
    teuthology.dispatcher.main(args)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/teuthology/dispatcher/__init__.py", line 61, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/teuthology/dispatcher/supervisor.py", line 59, in main
    verbose
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/teuthology/dispatcher/supervisor.py", line 152, in run_job
    unlock_targets(job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/teuthology/dispatcher/supervisor.py", line 207, in unlock_targets
    nuke(fake_ctx, True)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/teuthology/nuke/__init__.py", line 268, in nuke
    for unnuked in p:
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/teuthology/nuke/__init__.py", line 299, in nuke_one
    unlock_one(ctx, list(target.keys())[0], ctx.owner)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/teuthology/lock/ops.py", line 204, in unlock_one
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'unlock smithi086.front.sepia.ceph.com' reached maximum tries (10) after waiting for 32.5 seconds