2021-07-29T04:48:49.613 INFO:root:teuthology version: 1.1.0-73aa7e3b
2021-07-29T04:48:49.767 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi001.front.sepia.ceph.com' reimaging
2021-07-29T04:48:49.771 INFO:teuthology.lock.ops:Updating [ubuntu@smithi001.front.sepia.ceph.com]: reset os type and version on server
2021-07-29T04:48:49.771 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server
2021-07-29T04:48:49.790 INFO:teuthology.lock.ops:Node 'ubuntu@smithi001.front.sepia.ceph.com' reimaging is complete
2021-07-29T04:48:49.790 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi074.front.sepia.ceph.com' reimaging
2021-07-29T04:48:49.791 INFO:teuthology.lock.ops:Updating [ubuntu@smithi074.front.sepia.ceph.com]: reset os type and version on server
2021-07-29T04:48:49.791 INFO:teuthology.lock.ops:Updating smithi074.front.sepia.ceph.com on lock server
2021-07-29T04:48:49.807 INFO:teuthology.lock.ops:Node 'ubuntu@smithi074.front.sepia.ceph.com' reimaging is complete
2021-07-29T04:48:49.808 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi098.front.sepia.ceph.com' reimaging
2021-07-29T04:48:49.808 INFO:teuthology.lock.ops:Updating [ubuntu@smithi098.front.sepia.ceph.com]: reset os type and version on server
2021-07-29T04:48:49.808 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server
2021-07-29T04:48:49.827 INFO:teuthology.lock.ops:Node 'ubuntu@smithi098.front.sepia.ceph.com' reimaging is complete
2021-07-29T04:48:49.828 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi165.front.sepia.ceph.com' reimaging
2021-07-29T04:48:49.828 INFO:teuthology.lock.ops:Updating [ubuntu@smithi165.front.sepia.ceph.com]: reset os type and version on server
2021-07-29T04:48:49.829 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server
2021-07-29T04:48:49.845 INFO:teuthology.lock.ops:Node 'ubuntu@smithi165.front.sepia.ceph.com' reimaging is complete
2021-07-29T04:48:50.015 INFO:teuthology.provision.fog.smithi074:Scheduling deploy of centos 8.stream
2021-07-29T04:48:50.018 INFO:teuthology.provision.fog.smithi001:Scheduling deploy of centos 8.stream
2021-07-29T04:48:50.102 INFO:teuthology.provision.fog.smithi098:Scheduling deploy of centos 8.stream
2021-07-29T04:48:50.104 INFO:teuthology.provision.fog.smithi165:Scheduling deploy of centos 8.stream
2021-07-29T04:48:50.500 INFO:teuthology.orchestra.console:Power off smithi001
2021-07-29T04:48:50.536 INFO:teuthology.orchestra.console:Power off smithi074
2021-07-29T04:48:50.574 INFO:teuthology.orchestra.console:Power off smithi098
2021-07-29T04:48:50.633 INFO:teuthology.orchestra.console:Power off smithi165
2021-07-29T04:48:50.653 INFO:teuthology.orchestra.console:Power off for smithi001 completed
2021-07-29T04:48:50.687 INFO:teuthology.orchestra.console:Power off for smithi074 completed
2021-07-29T04:48:50.717 INFO:teuthology.orchestra.console:Power off for smithi098 completed
2021-07-29T04:48:50.755 INFO:teuthology.orchestra.console:Power on smithi001
2021-07-29T04:48:50.786 INFO:teuthology.orchestra.console:Power off for smithi165 completed
2021-07-29T04:48:50.790 INFO:teuthology.orchestra.console:Power on smithi074
2021-07-29T04:48:50.818 INFO:teuthology.orchestra.console:Power on smithi098
2021-07-29T04:48:50.888 INFO:teuthology.orchestra.console:Power on smithi165
2021-07-29T04:48:59.129 INFO:teuthology.orchestra.console:Power on for smithi001 completed
2021-07-29T04:48:59.156 INFO:teuthology.orchestra.console:Power on for smithi074 completed
2021-07-29T04:48:59.230 INFO:teuthology.provision.fog.smithi001:Waiting for deploy to finish
2021-07-29T04:48:59.253 INFO:teuthology.orchestra.console:Power on for smithi165 completed
2021-07-29T04:48:59.258 INFO:teuthology.provision.fog.smithi074:Waiting for deploy to finish
2021-07-29T04:48:59.280 INFO:teuthology.orchestra.console:Power on for smithi098 completed
2021-07-29T04:48:59.355 INFO:teuthology.provision.fog.smithi165:Waiting for deploy to finish
2021-07-29T04:48:59.381 INFO:teuthology.provision.fog.smithi098:Waiting for deploy to finish
2021-07-29T04:51:30.878 ERROR:teuthology.orchestra.connection:Error authenticating with smithi074.front.sepia.ceph.com: Authentication failed.
2021-07-29T04:53:16.507 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-29T04:53:16.795 INFO:teuthology.provision.fog.smithi001:Node is ready
2021-07-29T04:53:16.812 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001.front.sepia.ceph.com
2021-07-29T04:53:16.869 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1 smithi001.front.sepia.ceph.com smithi001
2021-07-29T04:53:17.216 INFO:teuthology.provision.fog.smithi001:Deploy complete!
2021-07-29T04:53:40.905 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-29T04:53:41.186 INFO:teuthology.provision.fog.smithi098:Node is ready
2021-07-29T04:53:41.203 INFO:teuthology.orchestra.run.smithi098.stdout:smithi098.front.sepia.ceph.com
2021-07-29T04:53:41.261 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98 smithi098.front.sepia.ceph.com smithi098
2021-07-29T04:53:41.605 INFO:teuthology.provision.fog.smithi098:Deploy complete!
2021-07-29T04:53:44.129 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-29T04:53:44.380 INFO:teuthology.provision.fog.smithi074:Node is ready
2021-07-29T04:53:44.437 INFO:teuthology.orchestra.run.smithi074.stdout:smithi074.front.sepia.ceph.com
2021-07-29T04:53:44.494 INFO:teuthology.orchestra.run.smithi074.stdout:172.21.15.74 smithi074.front.sepia.ceph.com smithi074
2021-07-29T04:53:44.840 INFO:teuthology.provision.fog.smithi074:Deploy complete!
2021-07-29T04:53:56.128 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-29T04:53:56.427 INFO:teuthology.provision.fog.smithi165:Node is ready
2021-07-29T04:53:56.443 INFO:teuthology.orchestra.run.smithi165.stdout:smithi165.front.sepia.ceph.com
2021-07-29T04:53:56.500 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.15.165 smithi165.front.sepia.ceph.com smithi165
2021-07-29T04:53:56.835 INFO:teuthology.provision.fog.smithi165:Deploy complete!
2021-07-29T04:53:57.039 INFO:teuthology.lock.ops:Checking smithi001.front.sepia.ceph.com
2021-07-29T04:53:57.039 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-29T04:53:57.056 INFO:teuthology.lock.ops:Checking smithi074.front.sepia.ceph.com
2021-07-29T04:53:57.056 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-29T04:53:57.070 INFO:teuthology.lock.ops:Checking smithi098.front.sepia.ceph.com
2021-07-29T04:53:57.071 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-29T04:53:57.085 INFO:teuthology.lock.ops:Checking smithi165.front.sepia.ceph.com
2021-07-29T04:53:57.085 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-29T04:53:57.099 INFO:teuthology.lock.ops:Updating [smithi001.front.sepia.ceph.com]: set os type and version on server
2021-07-29T04:53:57.099 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-29T04:53:57.348 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64
2021-07-29T04:53:57.404 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="CentOS Stream"
2021-07-29T04:53:57.405 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="8"
2021-07-29T04:53:57.405 INFO:teuthology.orchestra.run.smithi001.stdout:ID="centos"
2021-07-29T04:53:57.405 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE="rhel fedora"
2021-07-29T04:53:57.405 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="8"
2021-07-29T04:53:57.405 INFO:teuthology.orchestra.run.smithi001.stdout:PLATFORM_ID="platform:el8"
2021-07-29T04:53:57.406 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="CentOS Stream 8"
2021-07-29T04:53:57.406 INFO:teuthology.orchestra.run.smithi001.stdout:ANSI_COLOR="0;31"
2021-07-29T04:53:57.406 INFO:teuthology.orchestra.run.smithi001.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-07-29T04:53:57.406 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://centos.org/"
2021-07-29T04:53:57.407 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-07-29T04:53:57.407 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-07-29T04:53:57.407 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-07-29T04:53:57.408 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server
2021-07-29T04:53:57.425 INFO:teuthology.lock.ops:Updating [smithi074.front.sepia.ceph.com]: set os type and version on server
2021-07-29T04:53:57.425 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-29T04:53:57.659 INFO:teuthology.orchestra.run.smithi074.stdout:x86_64
2021-07-29T04:53:57.715 INFO:teuthology.orchestra.run.smithi074.stdout:NAME="CentOS Stream"
2021-07-29T04:53:57.715 INFO:teuthology.orchestra.run.smithi074.stdout:VERSION="8"
2021-07-29T04:53:57.716 INFO:teuthology.orchestra.run.smithi074.stdout:ID="centos"
2021-07-29T04:53:57.716 INFO:teuthology.orchestra.run.smithi074.stdout:ID_LIKE="rhel fedora"
2021-07-29T04:53:57.716 INFO:teuthology.orchestra.run.smithi074.stdout:VERSION_ID="8"
2021-07-29T04:53:57.716 INFO:teuthology.orchestra.run.smithi074.stdout:PLATFORM_ID="platform:el8"
2021-07-29T04:53:57.717 INFO:teuthology.orchestra.run.smithi074.stdout:PRETTY_NAME="CentOS Stream 8"
2021-07-29T04:53:57.717 INFO:teuthology.orchestra.run.smithi074.stdout:ANSI_COLOR="0;31"
2021-07-29T04:53:57.717 INFO:teuthology.orchestra.run.smithi074.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-07-29T04:53:57.717 INFO:teuthology.orchestra.run.smithi074.stdout:HOME_URL="https://centos.org/"
2021-07-29T04:53:57.718 INFO:teuthology.orchestra.run.smithi074.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-07-29T04:53:57.718 INFO:teuthology.orchestra.run.smithi074.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-07-29T04:53:57.718 INFO:teuthology.orchestra.run.smithi074.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-07-29T04:53:57.719 INFO:teuthology.lock.ops:Updating smithi074.front.sepia.ceph.com on lock server
2021-07-29T04:53:57.734 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: set os type and version on server
2021-07-29T04:53:57.735 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-29T04:53:57.966 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64
2021-07-29T04:53:58.022 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream"
2021-07-29T04:53:58.023 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8"
2021-07-29T04:53:58.023 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos"
2021-07-29T04:53:58.023 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora"
2021-07-29T04:53:58.023 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8"
2021-07-29T04:53:58.024 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8"
2021-07-29T04:53:58.024 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 8"
2021-07-29T04:53:58.024 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31"
2021-07-29T04:53:58.024 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-07-29T04:53:58.024 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/"
2021-07-29T04:53:58.025 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-07-29T04:53:58.025 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-07-29T04:53:58.025 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-07-29T04:53:58.026 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server
2021-07-29T04:53:58.042 INFO:teuthology.lock.ops:Updating [smithi165.front.sepia.ceph.com]: set os type and version on server
2021-07-29T04:53:58.042 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-29T04:53:58.271 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64
2021-07-29T04:53:58.327 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream"
2021-07-29T04:53:58.327 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="8"
2021-07-29T04:53:58.327 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos"
2021-07-29T04:53:58.328 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora"
2021-07-29T04:53:58.328 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="8"
2021-07-29T04:53:58.328 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el8"
2021-07-29T04:53:58.328 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 8"
2021-07-29T04:53:58.329 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31"
2021-07-29T04:53:58.329 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-07-29T04:53:58.329 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/"
2021-07-29T04:53:58.329 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-07-29T04:53:58.330 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-07-29T04:53:58.330 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-07-29T04:53:58.330 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server
2021-07-29T04:53:58.421 INFO:teuthology.dispatcher.supervisor:Running job 6300157
2021-07-29T04:53:58.422 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/virtualenv/bin/teuthology -v --owner scheduled_kchai@teuthology --archive /home/teuthworker/archive/kchai-2021-07-29_03:28:08-rados-wip-kefu-testing-2021-07-29-0930-distro-basic-smithi/6300157 --name kchai-2021-07-29_03:28:08-rados-wip-kefu-testing-2021-07-29-0930-distro-basic-smithi --description rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} -- /home/teuthworker/archive/kchai-2021-07-29_03:28:08-rados-wip-kefu-testing-2021-07-29-0930-distro-basic-smithi/6300157/orig.config.yaml
2021-07-29T04:53:58.425 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/kchai-2021-07-29_03:28:08-rados-wip-kefu-testing-2021-07-29-0930-distro-basic-smithi/6300157
2021-07-29T04:53:58.426 INFO:teuthology.dispatcher.supervisor:Job PID: 12357
2021-07-29T04:53:58.426 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-07-29T05:01:58.599 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-07-29T05:01:58.719 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-07-29T05:01:58.720 INFO:teuthology.nuke:Checking targets against current locks
2021-07-29T05:01:58.809 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/virtualenv/bin/teuthology-dispatcher", line 33, in <module>
    sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')())
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/scripts/dispatcher.py", line 33, in main
    teuthology.dispatcher.main(args)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/dispatcher/__init__.py", line 61, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/dispatcher/supervisor.py", line 59, in main
    verbose
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/dispatcher/supervisor.py", line 152, in run_job
    unlock_targets(job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/dispatcher/supervisor.py", line 196, in unlock_targets
    nuke(fake_ctx, True)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/nuke/__init__.py", line 250, in nuke
    if ctx.name not in lock['description']:
TypeError: argument of type 'NoneType' is not iterable