2021-07-25T02:45:39.847 INFO:root:teuthology version: 1.1.0-b6f3efab
2021-07-25T02:45:39.992 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi013.front.sepia.ceph.com' reimaging
2021-07-25T02:45:39.993 INFO:teuthology.lock.ops:Updating [ubuntu@smithi013.front.sepia.ceph.com]: reset os type and version on server
2021-07-25T02:45:39.993 INFO:teuthology.lock.ops:Updating smithi013.front.sepia.ceph.com on lock server
2021-07-25T02:45:40.009 INFO:teuthology.lock.ops:Node 'ubuntu@smithi013.front.sepia.ceph.com' reimaging is complete
2021-07-25T02:45:40.009 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi039.front.sepia.ceph.com' reimaging
2021-07-25T02:45:40.009 INFO:teuthology.lock.ops:Updating [ubuntu@smithi039.front.sepia.ceph.com]: reset os type and version on server
2021-07-25T02:45:40.010 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server
2021-07-25T02:45:40.024 INFO:teuthology.lock.ops:Node 'ubuntu@smithi039.front.sepia.ceph.com' reimaging is complete
2021-07-25T02:45:40.025 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging
2021-07-25T02:45:40.025 INFO:teuthology.lock.ops:Updating [ubuntu@smithi136.front.sepia.ceph.com]: reset os type and version on server
2021-07-25T02:45:40.025 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server
2021-07-25T02:45:40.039 INFO:teuthology.lock.ops:Node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging is complete
2021-07-25T02:45:40.040 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi174.front.sepia.ceph.com' reimaging
2021-07-25T02:45:40.040 INFO:teuthology.lock.ops:Updating [ubuntu@smithi174.front.sepia.ceph.com]: reset os type and version on server
2021-07-25T02:45:40.040 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server
2021-07-25T02:45:40.055 INFO:teuthology.lock.ops:Node 'ubuntu@smithi174.front.sepia.ceph.com' reimaging is complete
2021-07-25T02:45:40.249 INFO:teuthology.provision.fog.smithi013:Scheduling deploy of ubuntu 20.04
2021-07-25T02:45:40.253 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of ubuntu 20.04
2021-07-25T02:45:40.262 INFO:teuthology.provision.fog.smithi136:Scheduling deploy of ubuntu 20.04
2021-07-25T02:45:40.277 INFO:teuthology.provision.fog.smithi039:Scheduling deploy of ubuntu 20.04
2021-07-25T02:45:41.005 INFO:teuthology.orchestra.console:Power off smithi174
2021-07-25T02:45:41.019 INFO:teuthology.orchestra.console:Power off smithi039
2021-07-25T02:45:41.065 INFO:teuthology.orchestra.console:Power off smithi136
2021-07-25T02:45:41.093 INFO:teuthology.orchestra.console:Power off smithi013
2021-07-25T02:45:49.444 INFO:teuthology.orchestra.console:Power off for smithi136 completed
2021-07-25T02:45:49.451 INFO:teuthology.orchestra.console:Power off for smithi013 completed
2021-07-25T02:45:49.546 INFO:teuthology.orchestra.console:Power on smithi136
2021-07-25T02:45:49.554 INFO:teuthology.orchestra.console:Power on smithi013
2021-07-25T02:45:53.499 INFO:teuthology.orchestra.console:Power off for smithi174 completed
2021-07-25T02:45:53.504 INFO:teuthology.orchestra.console:Power off for smithi039 completed
2021-07-25T02:45:53.604 INFO:teuthology.orchestra.console:Power on smithi174
2021-07-25T02:45:53.612 INFO:teuthology.orchestra.console:Power on smithi039
2021-07-25T02:45:57.861 INFO:teuthology.orchestra.console:Power on for smithi174 completed
2021-07-25T02:45:57.907 INFO:teuthology.orchestra.console:Power on for smithi136 completed
2021-07-25T02:45:57.916 INFO:teuthology.orchestra.console:Power on for smithi013 completed
2021-07-25T02:45:57.963 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish
2021-07-25T02:45:58.009 INFO:teuthology.provision.fog.smithi136:Waiting for deploy to finish
2021-07-25T02:45:58.017 INFO:teuthology.provision.fog.smithi013:Waiting for deploy to finish
2021-07-25T02:46:01.977 INFO:teuthology.orchestra.console:Power on for smithi039 completed
2021-07-25T02:46:02.079 INFO:teuthology.provision.fog.smithi039:Waiting for deploy to finish
2021-07-25T02:51:57.180 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-25T02:51:57.653 INFO:teuthology.provision.fog.smithi013:Node is ready
2021-07-25T02:51:57.702 INFO:teuthology.orchestra.run.smithi013.stdout:smithi013.front.sepia.ceph.com
2021-07-25T02:51:57.710 INFO:teuthology.orchestra.run.smithi013.stdout:172.21.15.13 smithi013.front.sepia.ceph.com smithi013
2021-07-25T02:51:57.847 INFO:teuthology.provision.fog.smithi013:Deploy complete!
2021-07-25T02:52:09.642 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-25T02:52:10.137 INFO:teuthology.provision.fog.smithi136:Node is ready
2021-07-25T02:52:10.145 INFO:teuthology.orchestra.run.smithi136.stdout:smithi136.front.sepia.ceph.com
2021-07-25T02:52:10.195 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.15.136 smithi136.front.sepia.ceph.com smithi136
2021-07-25T02:52:10.408 INFO:teuthology.provision.fog.smithi136:Deploy complete!
2021-07-25T02:53:56.739 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-25T02:53:57.183 INFO:teuthology.provision.fog.smithi174:Node is ready
2021-07-25T02:53:57.231 INFO:teuthology.orchestra.run.smithi174.stdout:smithi174.front.sepia.ceph.com
2021-07-25T02:53:57.240 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.174 smithi174.front.sepia.ceph.com smithi174
2021-07-25T02:53:57.440 INFO:teuthology.provision.fog.smithi174:Deploy complete!
2021-07-25T02:55:23.405 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-25T02:55:23.859 INFO:teuthology.provision.fog.smithi039:Node is ready
2021-07-25T02:55:23.867 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039.front.sepia.ceph.com
2021-07-25T02:55:23.926 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39 smithi039.front.sepia.ceph.com smithi039
2021-07-25T02:55:24.166 INFO:teuthology.provision.fog.smithi039:Deploy complete!
2021-07-25T02:55:24.590 INFO:teuthology.lock.ops:Checking smithi013.front.sepia.ceph.com
2021-07-25T02:55:24.591 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-25T02:55:24.631 INFO:teuthology.lock.ops:Checking smithi039.front.sepia.ceph.com
2021-07-25T02:55:24.632 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-25T02:55:24.710 INFO:teuthology.lock.ops:Checking smithi136.front.sepia.ceph.com
2021-07-25T02:55:24.711 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-25T02:55:24.737 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com
2021-07-25T02:55:24.738 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-25T02:55:24.787 INFO:teuthology.lock.ops:Updating [smithi013.front.sepia.ceph.com]: set os type and version on server
2021-07-25T02:55:24.788 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-25T02:55:25.106 INFO:teuthology.orchestra.run.smithi013.stdout:x86_64
2021-07-25T02:55:25.143 INFO:teuthology.orchestra.run.smithi013.stdout:NAME="Ubuntu"
2021-07-25T02:55:25.153 INFO:teuthology.orchestra.run.smithi013.stdout:VERSION="20.04.2 LTS (Focal Fossa)"
2021-07-25T02:55:25.153 INFO:teuthology.orchestra.run.smithi013.stdout:ID=ubuntu
2021-07-25T02:55:25.153 INFO:teuthology.orchestra.run.smithi013.stdout:ID_LIKE=debian
2021-07-25T02:55:25.154 INFO:teuthology.orchestra.run.smithi013.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS"
2021-07-25T02:55:25.154 INFO:teuthology.orchestra.run.smithi013.stdout:VERSION_ID="20.04"
2021-07-25T02:55:25.154 INFO:teuthology.orchestra.run.smithi013.stdout:HOME_URL="https://www.ubuntu.com/"
2021-07-25T02:55:25.154 INFO:teuthology.orchestra.run.smithi013.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-07-25T02:55:25.155 INFO:teuthology.orchestra.run.smithi013.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-07-25T02:55:25.155 INFO:teuthology.orchestra.run.smithi013.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-07-25T02:55:25.155 INFO:teuthology.orchestra.run.smithi013.stdout:VERSION_CODENAME=focal
2021-07-25T02:55:25.155 INFO:teuthology.orchestra.run.smithi013.stdout:UBUNTU_CODENAME=focal
2021-07-25T02:55:25.165 INFO:teuthology.lock.ops:Updating smithi013.front.sepia.ceph.com on lock server
2021-07-25T02:55:25.238 INFO:teuthology.lock.ops:Updating [smithi039.front.sepia.ceph.com]: set os type and version on server
2021-07-25T02:55:25.238 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-25T02:55:25.516 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64
2021-07-25T02:55:25.563 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="Ubuntu"
2021-07-25T02:55:25.564 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="20.04.2 LTS (Focal Fossa)"
2021-07-25T02:55:25.564 INFO:teuthology.orchestra.run.smithi039.stdout:ID=ubuntu
2021-07-25T02:55:25.564 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE=debian
2021-07-25T02:55:25.564 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS"
2021-07-25T02:55:25.565 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="20.04"
2021-07-25T02:55:25.565 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://www.ubuntu.com/"
2021-07-25T02:55:25.565 INFO:teuthology.orchestra.run.smithi039.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-07-25T02:55:25.565 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-07-25T02:55:25.565 INFO:teuthology.orchestra.run.smithi039.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-07-25T02:55:25.566 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_CODENAME=focal
2021-07-25T02:55:25.566 INFO:teuthology.orchestra.run.smithi039.stdout:UBUNTU_CODENAME=focal
2021-07-25T02:55:25.566 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server
2021-07-25T02:55:25.585 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: set os type and version on server
2021-07-25T02:55:25.585 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-25T02:55:25.804 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64
2021-07-25T02:55:25.833 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="Ubuntu"
2021-07-25T02:55:25.834 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="20.04.2 LTS (Focal Fossa)"
2021-07-25T02:55:25.835 INFO:teuthology.orchestra.run.smithi136.stdout:ID=ubuntu
2021-07-25T02:55:25.835 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE=debian
2021-07-25T02:55:25.836 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS"
2021-07-25T02:55:25.836 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="20.04"
2021-07-25T02:55:25.836 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.ubuntu.com/"
2021-07-25T02:55:25.837 INFO:teuthology.orchestra.run.smithi136.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-07-25T02:55:25.837 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-07-25T02:55:25.837 INFO:teuthology.orchestra.run.smithi136.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-07-25T02:55:25.837 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_CODENAME=focal
2021-07-25T02:55:25.838 INFO:teuthology.orchestra.run.smithi136.stdout:UBUNTU_CODENAME=focal
2021-07-25T02:55:25.838 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server
2021-07-25T02:55:25.857 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server
2021-07-25T02:55:25.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-25T02:55:26.111 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64
2021-07-25T02:55:26.119 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="Ubuntu"
2021-07-25T02:55:26.119 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="20.04.2 LTS (Focal Fossa)"
2021-07-25T02:55:26.120 INFO:teuthology.orchestra.run.smithi174.stdout:ID=ubuntu
2021-07-25T02:55:26.120 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE=debian
2021-07-25T02:55:26.120 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS"
2021-07-25T02:55:26.120 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="20.04"
2021-07-25T02:55:26.120 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://www.ubuntu.com/"
2021-07-25T02:55:26.121 INFO:teuthology.orchestra.run.smithi174.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-07-25T02:55:26.121 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-07-25T02:55:26.121 INFO:teuthology.orchestra.run.smithi174.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-07-25T02:55:26.121 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_CODENAME=focal
2021-07-25T02:55:26.122 INFO:teuthology.orchestra.run.smithi174.stdout:UBUNTU_CODENAME=focal
2021-07-25T02:55:26.122 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server
2021-07-25T02:55:26.296 INFO:teuthology.dispatcher.supervisor:Running job 6290660
2021-07-25T02:55:26.296 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/virtualenv/bin/teuthology -v --owner scheduled_kchai@teuthology --archive /home/teuthworker/archive/kchai-2021-07-25_02:43:47-rados-wip-kefu-testing-2021-07-24-2153-distro-basic-smithi/6290660 --name kchai-2021-07-25_02:43:47-rados-wip-kefu-testing-2021-07-24-2153-distro-basic-smithi --description rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} -- /home/teuthworker/archive/kchai-2021-07-25_02:43:47-rados-wip-kefu-testing-2021-07-24-2153-distro-basic-smithi/6290660/orig.config.yaml
2021-07-25T02:55:26.301 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/kchai-2021-07-25_02:43:47-rados-wip-kefu-testing-2021-07-24-2153-distro-basic-smithi/6290660
2021-07-25T02:55:26.302 INFO:teuthology.dispatcher.supervisor:Job PID: 13747
2021-07-25T02:55:26.302 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-07-25T03:11:28.011 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-07-25T03:11:28.261 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-07-25T03:11:28.261 INFO:teuthology.nuke:Checking targets against current locks
2021-07-25T03:11:28.335 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/virtualenv/bin/teuthology-dispatcher", line 33, in <module>
    sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')())
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/scripts/dispatcher.py", line 33, in main
    teuthology.dispatcher.main(args)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/__init__.py", line 61, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 59, in main
    verbose
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 152, in run_job
    unlock_targets(job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 196, in unlock_targets
    nuke(fake_ctx, True)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/nuke/__init__.py", line 250, in nuke
    if ctx.name not in lock['description']:
TypeError: argument of type 'NoneType' is not iterable