2022-07-20T09:48:58.023 INFO:root:teuthology version: 1.1.1.dev383+g89d874d5 2022-07-20T09:48:58.178 INFO:teuthology.lock.ops:Start node 'smithi170.front.sepia.ceph.com' reimaging 2022-07-20T09:48:58.180 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: reset os type and version on server 2022-07-20T09:48:58.181 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2022-07-20T09:48:58.207 INFO:teuthology.lock.ops:Node 'smithi170.front.sepia.ceph.com' reimaging is complete 2022-07-20T09:48:58.207 INFO:teuthology.lock.ops:Start node 'smithi186.front.sepia.ceph.com' reimaging 2022-07-20T09:48:58.208 INFO:teuthology.lock.ops:Updating [smithi186.front.sepia.ceph.com]: reset os type and version on server 2022-07-20T09:48:58.208 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2022-07-20T09:48:58.230 INFO:teuthology.lock.ops:Node 'smithi186.front.sepia.ceph.com' reimaging is complete 2022-07-20T09:48:58.446 INFO:teuthology.provision.fog.smithi170:Scheduling deploy of centos 8.stream 2022-07-20T09:48:58.453 INFO:teuthology.provision.fog.smithi186:Scheduling deploy of centos 8.stream 2022-07-20T09:48:58.802 INFO:teuthology.orchestra.console:Power off smithi170 2022-07-20T09:48:58.877 INFO:teuthology.orchestra.console:Power off smithi186 2022-07-20T09:49:07.204 INFO:teuthology.orchestra.console:Power off for smithi170 completed 2022-07-20T09:49:07.260 INFO:teuthology.orchestra.console:Power off for smithi186 completed 2022-07-20T09:49:07.307 INFO:teuthology.orchestra.console:Power on smithi170 2022-07-20T09:49:07.362 INFO:teuthology.orchestra.console:Power on smithi186 2022-07-20T09:49:15.685 INFO:teuthology.orchestra.console:Power on for smithi170 completed 2022-07-20T09:49:15.738 INFO:teuthology.orchestra.console:Power on for smithi186 completed 2022-07-20T09:49:15.788 INFO:teuthology.provision.fog.smithi170:Waiting for deploy to finish 2022-07-20T09:49:15.839 INFO:teuthology.provision.fog.smithi186:Waiting for deploy to finish 2022-07-20T09:53:22.519 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-20T09:53:47.310 INFO:teuthology.provision.fog.smithi170:Node is ready 2022-07-20T09:53:47.333 INFO:teuthology.orchestra.run.smithi170.stdout:smithi170.front.sepia.ceph.com 2022-07-20T09:53:47.392 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170 smithi170.front.sepia.ceph.com smithi170 2022-07-20T09:53:47.770 INFO:teuthology.provision.fog.smithi170:Deploy complete! 2022-07-20T09:54:02.969 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-07-20T09:54:03.231 INFO:teuthology.provision.fog.smithi186:Node is ready 2022-07-20T09:54:03.250 INFO:teuthology.orchestra.run.smithi186.stdout:smithi186.front.sepia.ceph.com 2022-07-20T09:54:03.308 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.15.186 smithi186.front.sepia.ceph.com smithi186 2022-07-20T09:54:03.671 INFO:teuthology.provision.fog.smithi186:Deploy complete! 2022-07-20T09:54:03.888 INFO:teuthology.lock.ops:Checking smithi170.front.sepia.ceph.com 2022-07-20T09:54:03.889 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-20T09:54:03.925 INFO:teuthology.lock.ops:Checking smithi186.front.sepia.ceph.com 2022-07-20T09:54:03.926 INFO:teuthology.lock.ops:New key found. Updating... 2022-07-20T09:54:03.953 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: set os type and version on server 2022-07-20T09:54:03.954 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2022-07-20T09:54:04.230 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2022-07-20T09:54:04.288 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream" 2022-07-20T09:54:04.289 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8" 2022-07-20T09:54:04.289 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos" 2022-07-20T09:54:04.290 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora" 2022-07-20T09:54:04.290 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8" 2022-07-20T09:54:04.291 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8" 2022-07-20T09:54:04.292 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-20T09:54:04.292 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2022-07-20T09:54:04.293 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-20T09:54:04.293 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/" 2022-07-20T09:54:04.293 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-20T09:54:04.294 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-20T09:54:04.294 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-20T09:54:04.297 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2022-07-20T09:54:04.318 INFO:teuthology.lock.ops:Updating [smithi186.front.sepia.ceph.com]: set os type and version on server 2022-07-20T09:54:04.319 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2022-07-20T09:54:04.521 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2022-07-20T09:54:04.580 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="CentOS Stream" 2022-07-20T09:54:04.580 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="8" 2022-07-20T09:54:04.580 INFO:teuthology.orchestra.run.smithi186.stdout:ID="centos" 2022-07-20T09:54:04.581 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="rhel fedora" 2022-07-20T09:54:04.581 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="8" 2022-07-20T09:54:04.582 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el8" 2022-07-20T09:54:04.582 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="CentOS Stream 8" 2022-07-20T09:54:04.583 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2022-07-20T09:54:04.584 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-07-20T09:54:04.584 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://centos.org/" 2022-07-20T09:54:04.585 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-07-20T09:54:04.585 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-07-20T09:54:04.586 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-07-20T09:54:04.586 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2022-07-20T09:54:04.686 INFO:teuthology.dispatcher.supervisor:Running job 6939130 2022-07-20T09:54:04.687 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2022-07-19_23:27:22-orch:cephadm-wip-adk2-testing-2022-07-19-1528-distro-default-smithi/6939130 --name adking-2022-07-19_23:27:22-orch:cephadm-wip-adk2-testing-2022-07-19-1528-distro-default-smithi --description orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} -- /home/teuthworker/archive/adking-2022-07-19_23:27:22-orch:cephadm-wip-adk2-testing-2022-07-19-1528-distro-default-smithi/6939130/orig.config.yaml 2022-07-20T09:54:04.692 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2022-07-19_23:27:22-orch:cephadm-wip-adk2-testing-2022-07-19-1528-distro-default-smithi/6939130 2022-07-20T09:54:04.693 INFO:teuthology.dispatcher.supervisor:Job PID: 12231 2022-07-20T09:54:04.693 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-07-20T21:55:05.263 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-07-20T21:55:05.427 INFO:teuthology.kill:Killing Pids: {12231} 2022-07-20T21:55:07.088 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-07-20T21:55:07.089 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-07-20T21:55:07.089 INFO:teuthology.misc:Compressing logs... 2022-07-20T21:55:07.090 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2022-07-20T21:55:07.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2022-07-20T21:55:07.370 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-07-20T21:55:07.509 INFO:teuthology.orchestra.run.smithi186.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-07-20T21:55:07.543 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-07-20T21:55:07.543 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now 2022-07-20T21:55:07.593 INFO:teuthology.orchestra.run.smithi186.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-07-20T21:55:07.594 INFO:teuthology.orchestra.run.smithi186.stderr:tar: Error is not recoverable: exiting now 2022-07-20T21:55:07.597 INFO:teuthology.misc:Compressing logs... 2022-07-20T21:55:08.002 INFO:teuthology.orchestra.run.smithi186.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-07-20T21:55:08.734 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-07-20T21:55:09.220 INFO:teuthology.misc:Compressing logs... 2022-07-20T21:55:15.934 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/cephadm.log: file size changed while zipping 2022-07-20T21:55:19.277 INFO:teuthology.orchestra.run.smithi186.stderr:gzip: /var/log/ceph/e663965c-0812-11ed-842e-001a4aab830c/ceph-mon.smithi186.log: file size changed while zipping 2022-07-20T21:55:19.635 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/e663965c-0812-11ed-842e-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-07-20T21:55:23.142 INFO:teuthology.orchestra.run.smithi186.stderr:gzip: /var/log/ceph/e663965c-0812-11ed-842e-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-07-20T21:55:26.738 INFO:teuthology.orchestra.run.smithi186.stderr:gzip: /var/log/ceph/e663965c-0812-11ed-842e-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-07-20T21:55:30.173 INFO:teuthology.orchestra.run.smithi186.stderr:gzip: /var/log/ceph/e663965c-0812-11ed-842e-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-07-20T21:55:34.290 INFO:teuthology.orchestra.run.smithi186.stderr:gzip: /var/log/ceph/e663965c-0812-11ed-842e-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-07-20T21:55:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/e663965c-0812-11ed-842e-001a4aab830c/ceph-mon.smithi170.log: file size changed while zipping 2022-07-20T21:56:09.266 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/e663965c-0812-11ed-842e-001a4aab830c/ceph-mgr.smithi170.rwxexo.log: file size changed while zipping 2022-07-20T21:56:12.925 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/e663965c-0812-11ed-842e-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-07-20T21:56:17.138 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/e663965c-0812-11ed-842e-001a4aab830c/ceph-volume.log: file size changed while zipping 2022-07-20T21:56:22.207 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/e663965c-0812-11ed-842e-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-07-20T21:56:43.733 INFO:teuthology.orchestra.run.smithi170.stderr:tar: ./e663965c-0812-11ed-842e-001a4aab830c: file changed as we read it 2022-07-20T21:56:49.433 INFO:teuthology.kill:No teuthology processes running 2022-07-20T21:56:51.003 INFO:teuthology.kill:Nuking machines: ['smithi170', 'smithi186'] 2022-07-20T21:56:52.154 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-07-20T21:56:52.155 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-07-20T21:56:52.155 INFO:teuthology.kill:2022-07-20 21:56:52,154.154 INFO:teuthology.nuke:targets: 2022-07-20T21:56:52.156 INFO:teuthology.kill: smithi170.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQy5+tLRsD4YzY8UNsXzSI8VmlqnG/Pw7svjSjR+hF6V5UM+vyBGtaVvnq1TE400+ClIloRpgl3U7UJRGJw3K6Pzqnk2z04UL/P1NAapA5AeKL9aEZoaOJb4U8dWKcpOsYxyKBUVkKmHPfHuRbYXLpEP3u8pY2WBj+cBGyYp1evhV8LSiInFLAoeMRsN/bv8ZtvmI3IJTG1CUvOBY87Mu0gBleSYuI8UzEjKpzJF1h7AKOUgREjBTHV0cPTiHYKEFCJFYqgGNtJAw60JpkMQYn3gyhOq/WeIGa89buxq84I6KcCLJbeOPuoPYgtuyo/CyKasTaBXoGvPiZsRaFbpyq8M5PmM9WV1sSz9SV/3ngb3u9Ju65jq8HEltYFXU5iHPtJ4LQMT5t7/eiBKXX84VMNcA531jBsnVrXExTmENV1TAe+aiHXYMAkLlEsiqX8A7PGtUwk8UMAep+y3axfUxbBmiS+uNbJm2bOt2CGUc3WoKUsRDlHQKqJoenIgCF1l8= 2022-07-20T21:56:52.156 INFO:teuthology.kill: smithi186.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6h121x4LKj7vmTulLE/2TK4y07kaXBqPu29jBYnHBz0MZHakAX+OEIZKH6CuZ6tsBO7naiRzUNcdtTcJAiHgn/ApW4+Y/NMgGY8y+XT9Q3TlVQN09d5q9WIKAK8s85vDSbHeDiBv2J/bpYRzistjnGD7lMCLCKzWE2fmqNzC7idQS2GPQ+IGeKeR5z9dgItfOK+AJCqfDNsD6a9HOmWG132sTgYQKhIxUeDCeJD9Wdjf11HUHsyMGNfPasLP0zG+srXY+k6q8bbNBZ+tXiwuT0nDI3dq+zytPuHLeEOrqtdiKyh3YAZsu6OyYNTSAaHqzByjpX+oJ1NSJ6R/iS9euvWaPbqRJVfQQ+pkvDfOGg17Lkc/YSaI3uR5JAhh3AmJ7Yni1X+rFaHgGVvirs6C31tmmClYuAekjn2aeoWucxbMHmIFUCyJbYjBC1ePQsQoHg/zrTMtNLV/3zZ7/Lk3y9MJ+8vr1vU3lZXmPUWhF09nhJTFNi8hSk5zP2MAoDRU= 2022-07-20T21:56:52.157 INFO:teuthology.kill:2022-07-20 21:56:52,154.154 INFO:teuthology.nuke:Checking targets against current locks 2022-07-20T21:56:52.216 INFO:teuthology.kill:2022-07-20 21:56:52,216.216 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-20T21:56:52.240 INFO:teuthology.kill:2022-07-20 21:56:52,240.240 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-20T21:56:52.304 INFO:teuthology.kill:2022-07-20 21:56:52,303.303 INFO:teuthology.orchestra.console:Power off smithi170 2022-07-20T21:56:52.344 INFO:teuthology.kill:2022-07-20 21:56:52,343.343 INFO:teuthology.orchestra.console:Power off smithi186 2022-07-20T21:57:00.680 INFO:teuthology.kill:2022-07-20 21:57:00,679.679 INFO:teuthology.orchestra.console:Power off for smithi170 completed 2022-07-20T21:57:00.715 INFO:teuthology.kill:2022-07-20 21:57:00,714.714 INFO:teuthology.orchestra.console:Power off for smithi186 completed 2022-07-20T21:57:00.852 INFO:teuthology.kill:2022-07-20 21:57:00,851.851 INFO:teuthology.lock.ops:unlocked: smithi170.front.sepia.ceph.com 2022-07-20T21:57:00.874 INFO:teuthology.kill:2022-07-20 21:57:00,874.874 INFO:teuthology.lock.ops:unlocked: smithi186.front.sepia.ceph.com 2022-07-20T21:59:01.247 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-07-20T21:59:01.369 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi186 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-07-20_14:54:27-krbd-octopus-release-testing-default-smithi/6940776