2021-07-17T05:50:38.190 INFO:root:teuthology version: 1.1.0-b6f3efab 2021-07-17T05:50:38.328 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi012.front.sepia.ceph.com' reimaging 2021-07-17T05:50:38.330 INFO:teuthology.lock.ops:Updating [ubuntu@smithi012.front.sepia.ceph.com]: reset os type and version on server 2021-07-17T05:50:38.330 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2021-07-17T05:50:38.367 INFO:teuthology.lock.ops:Node 'ubuntu@smithi012.front.sepia.ceph.com' reimaging is complete 2021-07-17T05:50:38.367 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi163.front.sepia.ceph.com' reimaging 2021-07-17T05:50:38.368 INFO:teuthology.lock.ops:Updating [ubuntu@smithi163.front.sepia.ceph.com]: reset os type and version on server 2021-07-17T05:50:38.368 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2021-07-17T05:50:38.382 INFO:teuthology.lock.ops:Node 'ubuntu@smithi163.front.sepia.ceph.com' reimaging is complete 2021-07-17T05:50:38.383 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi164.front.sepia.ceph.com' reimaging 2021-07-17T05:50:38.383 INFO:teuthology.lock.ops:Updating [ubuntu@smithi164.front.sepia.ceph.com]: reset os type and version on server 2021-07-17T05:50:38.384 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2021-07-17T05:50:38.397 INFO:teuthology.lock.ops:Node 'ubuntu@smithi164.front.sepia.ceph.com' reimaging is complete 2021-07-17T05:50:38.577 INFO:teuthology.provision.fog.smithi012:Scheduling deploy of centos 8.stream 2021-07-17T05:50:38.583 INFO:teuthology.provision.fog.smithi163:Scheduling deploy of centos 8.stream 2021-07-17T05:50:38.617 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of centos 8.stream 2021-07-17T05:50:38.946 INFO:teuthology.orchestra.console:Power off smithi163 2021-07-17T05:50:39.045 INFO:teuthology.orchestra.console:Power off smithi164 2021-07-17T05:50:39.062 INFO:teuthology.orchestra.console:Power off smithi012 2021-07-17T05:50:47.416 INFO:teuthology.orchestra.console:Power off for smithi164 completed 2021-07-17T05:50:47.424 INFO:teuthology.orchestra.console:Power off for smithi012 completed 2021-07-17T05:50:47.518 INFO:teuthology.orchestra.console:Power on smithi164 2021-07-17T05:50:47.528 INFO:teuthology.orchestra.console:Power on smithi012 2021-07-17T05:50:51.435 INFO:teuthology.orchestra.console:Power off for smithi163 completed 2021-07-17T05:50:51.537 INFO:teuthology.orchestra.console:Power on smithi163 2021-07-17T05:50:55.894 INFO:teuthology.orchestra.console:Power on for smithi012 completed 2021-07-17T05:50:55.962 INFO:teuthology.orchestra.console:Power on for smithi164 completed 2021-07-17T05:50:55.995 INFO:teuthology.provision.fog.smithi012:Waiting for deploy to finish 2021-07-17T05:50:56.064 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish 2021-07-17T05:50:59.908 INFO:teuthology.orchestra.console:Power on for smithi163 completed 2021-07-17T05:51:00.010 INFO:teuthology.provision.fog.smithi163:Waiting for deploy to finish 2021-07-17T05:55:52.251 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-17T05:55:52.545 INFO:teuthology.provision.fog.smithi164:Node is ready 2021-07-17T05:55:52.562 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164.front.sepia.ceph.com 2021-07-17T05:55:52.619 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164 2021-07-17T05:55:52.953 INFO:teuthology.provision.fog.smithi164:Deploy complete! 2021-07-17T05:56:19.239 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-17T05:56:19.531 INFO:teuthology.provision.fog.smithi012:Node is ready 2021-07-17T05:56:19.548 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012.front.sepia.ceph.com 2021-07-17T05:56:19.607 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12 smithi012.front.sepia.ceph.com smithi012 2021-07-17T05:56:19.937 INFO:teuthology.provision.fog.smithi012:Deploy complete! 2021-07-17T05:56:20.208 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-17T05:56:20.458 INFO:teuthology.provision.fog.smithi163:Node is ready 2021-07-17T05:56:20.517 INFO:teuthology.orchestra.run.smithi163.stdout:smithi163.front.sepia.ceph.com 2021-07-17T05:56:20.575 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.15.163 smithi163.front.sepia.ceph.com smithi163 2021-07-17T05:56:20.924 INFO:teuthology.provision.fog.smithi163:Deploy complete! 2021-07-17T05:56:21.071 INFO:teuthology.lock.ops:Checking smithi012.front.sepia.ceph.com 2021-07-17T05:56:21.072 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-17T05:56:21.088 INFO:teuthology.lock.ops:Checking smithi163.front.sepia.ceph.com 2021-07-17T05:56:21.088 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-17T05:56:21.103 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com 2021-07-17T05:56:21.103 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-17T05:56:21.123 INFO:teuthology.lock.ops:Updating [smithi012.front.sepia.ceph.com]: set os type and version on server 2021-07-17T05:56:21.123 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-17T05:56:21.319 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2021-07-17T05:56:21.378 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="CentOS Stream" 2021-07-17T05:56:21.379 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="8" 2021-07-17T05:56:21.379 INFO:teuthology.orchestra.run.smithi012.stdout:ID="centos" 2021-07-17T05:56:21.380 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="rhel fedora" 2021-07-17T05:56:21.380 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="8" 2021-07-17T05:56:21.380 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el8" 2021-07-17T05:56:21.380 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="CentOS Stream 8" 2021-07-17T05:56:21.381 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2021-07-17T05:56:21.381 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-07-17T05:56:21.382 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://centos.org/" 2021-07-17T05:56:21.382 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-07-17T05:56:21.382 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-07-17T05:56:21.382 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-07-17T05:56:21.384 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2021-07-17T05:56:21.401 INFO:teuthology.lock.ops:Updating [smithi163.front.sepia.ceph.com]: set os type and version on server 2021-07-17T05:56:21.402 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-17T05:56:21.629 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2021-07-17T05:56:21.686 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Stream" 2021-07-17T05:56:21.686 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="8" 2021-07-17T05:56:21.686 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos" 2021-07-17T05:56:21.686 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora" 2021-07-17T05:56:21.687 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="8" 2021-07-17T05:56:21.687 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el8" 2021-07-17T05:56:21.687 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Stream 8" 2021-07-17T05:56:21.687 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2021-07-17T05:56:21.688 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-07-17T05:56:21.688 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://centos.org/" 2021-07-17T05:56:21.688 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-07-17T05:56:21.688 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-07-17T05:56:21.688 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-07-17T05:56:21.692 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2021-07-17T05:56:21.706 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server 2021-07-17T05:56:21.707 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-17T05:56:21.942 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2021-07-17T05:56:21.998 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2021-07-17T05:56:21.998 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8" 2021-07-17T05:56:21.999 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2021-07-17T05:56:21.999 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2021-07-17T05:56:21.999 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8" 2021-07-17T05:56:21.999 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8" 2021-07-17T05:56:21.999 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 8" 2021-07-17T05:56:22.000 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2021-07-17T05:56:22.000 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-07-17T05:56:22.000 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2021-07-17T05:56:22.000 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-07-17T05:56:22.000 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-07-17T05:56:22.001 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-07-17T05:56:22.001 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2021-07-17T05:56:22.081 INFO:teuthology.dispatcher.supervisor:Running job 6277153 2021-07-17T05:56:22.081 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/virtualenv/bin/teuthology -v --owner scheduled_kchai@teuthology --archive /home/teuthworker/archive/kchai-2021-07-17_04:37:01-rados-wip-kefu-testing-2021-07-17-1006-distro-basic-smithi/6277153 --name kchai-2021-07-17_04:37:01-rados-wip-kefu-testing-2021-07-17-1006-distro-basic-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8.stream} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/kchai-2021-07-17_04:37:01-rados-wip-kefu-testing-2021-07-17-1006-distro-basic-smithi/6277153/orig.config.yaml 2021-07-17T05:56:22.085 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/kchai-2021-07-17_04:37:01-rados-wip-kefu-testing-2021-07-17-1006-distro-basic-smithi/6277153 2021-07-17T05:56:22.086 INFO:teuthology.dispatcher.supervisor:Job PID: 24425 2021-07-17T05:56:22.086 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-07-17T15:51:02.413 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-07-17T15:51:02.603 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-07-17T15:51:02.603 INFO:teuthology.nuke:Checking targets against current locks 2021-07-17T15:51:02.699 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 59, in main verbose File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 152, in run_job unlock_targets(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 196, in unlock_targets nuke(fake_ctx, True) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/nuke/__init__.py", line 250, in nuke if ctx.name not in lock['description']: TypeError: argument of type 'NoneType' is not iterable