2022-05-11T07:40:05.962 INFO:root:teuthology version: 1.1.1.dev333+gb018fa01 2022-05-11T07:40:06.087 INFO:teuthology.lock.ops:Start node 'smithi131.front.sepia.ceph.com' reimaging 2022-05-11T07:40:06.088 INFO:teuthology.lock.ops:Updating [smithi131.front.sepia.ceph.com]: reset os type and version on server 2022-05-11T07:40:06.089 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2022-05-11T07:40:06.142 INFO:teuthology.lock.ops:Node 'smithi131.front.sepia.ceph.com' reimaging is complete 2022-05-11T07:40:06.143 INFO:teuthology.lock.ops:Start node 'smithi139.front.sepia.ceph.com' reimaging 2022-05-11T07:40:06.143 INFO:teuthology.lock.ops:Updating [smithi139.front.sepia.ceph.com]: reset os type and version on server 2022-05-11T07:40:06.143 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2022-05-11T07:40:06.162 INFO:teuthology.lock.ops:Node 'smithi139.front.sepia.ceph.com' reimaging is complete 2022-05-11T07:40:06.382 INFO:teuthology.provision.fog.smithi139:Scheduling deploy of rhel 8.5 2022-05-11T07:40:06.399 INFO:teuthology.provision.fog.smithi131:Scheduling deploy of rhel 8.5 2022-05-11T07:40:06.840 INFO:teuthology.orchestra.console:Power off smithi131 2022-05-11T07:40:06.854 INFO:teuthology.orchestra.console:Power off smithi139 2022-05-11T07:40:06.981 INFO:teuthology.orchestra.console:Power off for smithi131 completed 2022-05-11T07:40:06.991 INFO:teuthology.orchestra.console:Power off for smithi139 completed 2022-05-11T07:40:07.083 INFO:teuthology.orchestra.console:Power on smithi131 2022-05-11T07:40:07.092 INFO:teuthology.orchestra.console:Power on smithi139 2022-05-11T07:40:11.340 INFO:teuthology.orchestra.console:Power on for smithi139 completed 2022-05-11T07:40:11.441 INFO:teuthology.provision.fog.smithi139:Waiting for deploy to finish 2022-05-11T07:40:15.461 INFO:teuthology.orchestra.console:Power on for smithi131 completed 2022-05-11T07:40:15.562 INFO:teuthology.provision.fog.smithi131:Waiting for deploy to finish 2022-05-11T07:44:48.566 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-05-11T07:44:48.792 INFO:teuthology.provision.fog.smithi131:Node is ready 2022-05-11T07:44:48.808 INFO:teuthology.orchestra.run.smithi131.stdout:smithi131.front.sepia.ceph.com 2022-05-11T07:44:48.865 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131 smithi131.front.sepia.ceph.com smithi131 2022-05-11T07:44:49.185 INFO:teuthology.provision.fog.smithi131:Deploy complete! 2022-05-11T07:45:39.504 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-05-11T07:45:39.745 INFO:teuthology.provision.fog.smithi139:Node is ready 2022-05-11T07:45:39.761 INFO:teuthology.orchestra.run.smithi139.stdout:smithi139.front.sepia.ceph.com 2022-05-11T07:45:39.817 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139 smithi139.front.sepia.ceph.com smithi139 2022-05-11T07:45:40.145 INFO:teuthology.provision.fog.smithi139:Deploy complete! 2022-05-11T07:45:40.288 INFO:teuthology.lock.ops:Checking smithi131.front.sepia.ceph.com 2022-05-11T07:45:40.288 INFO:teuthology.lock.ops:New key found. Updating... 2022-05-11T07:45:40.306 INFO:teuthology.lock.ops:Checking smithi139.front.sepia.ceph.com 2022-05-11T07:45:40.307 INFO:teuthology.lock.ops:New key found. Updating... 2022-05-11T07:45:40.324 INFO:teuthology.lock.ops:Updating [smithi131.front.sepia.ceph.com]: set os type and version on server 2022-05-11T07:45:40.325 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2022-05-11T07:45:40.570 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2022-05-11T07:45:40.626 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="Red Hat Enterprise Linux" 2022-05-11T07:45:40.626 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="8.5 (Ootpa)" 2022-05-11T07:45:40.627 INFO:teuthology.orchestra.run.smithi131.stdout:ID="rhel" 2022-05-11T07:45:40.627 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="fedora" 2022-05-11T07:45:40.627 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="8.5" 2022-05-11T07:45:40.627 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el8" 2022-05-11T07:45:40.628 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.5 (Ootpa)" 2022-05-11T07:45:40.628 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2022-05-11T07:45:40.628 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-05-11T07:45:40.628 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://www.redhat.com/" 2022-05-11T07:45:40.628 INFO:teuthology.orchestra.run.smithi131.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-05-11T07:45:40.629 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-05-11T07:45:40.629 INFO:teuthology.orchestra.run.smithi131.stdout: 2022-05-11T07:45:40.629 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-05-11T07:45:40.629 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.5 2022-05-11T07:45:40.629 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-05-11T07:45:40.630 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.5" 2022-05-11T07:45:40.631 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2022-05-11T07:45:40.647 INFO:teuthology.lock.ops:Updating [smithi139.front.sepia.ceph.com]: set os type and version on server 2022-05-11T07:45:40.647 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2022-05-11T07:45:40.835 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2022-05-11T07:45:40.892 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="Red Hat Enterprise Linux" 2022-05-11T07:45:40.892 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="8.5 (Ootpa)" 2022-05-11T07:45:40.892 INFO:teuthology.orchestra.run.smithi139.stdout:ID="rhel" 2022-05-11T07:45:40.893 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="fedora" 2022-05-11T07:45:40.893 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="8.5" 2022-05-11T07:45:40.893 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el8" 2022-05-11T07:45:40.893 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.5 (Ootpa)" 2022-05-11T07:45:40.893 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2022-05-11T07:45:40.894 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-05-11T07:45:40.894 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://www.redhat.com/" 2022-05-11T07:45:40.894 INFO:teuthology.orchestra.run.smithi139.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-05-11T07:45:40.894 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-05-11T07:45:40.895 INFO:teuthology.orchestra.run.smithi139.stdout: 2022-05-11T07:45:40.895 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-05-11T07:45:40.895 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.5 2022-05-11T07:45:40.895 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-05-11T07:45:40.895 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.5" 2022-05-11T07:45:40.896 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2022-05-11T07:45:40.992 INFO:teuthology.dispatcher.supervisor:Running job 6830310 2022-05-11T07:45:40.992 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b018fa018d3f885cbc018f839b42536477653f84/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2022-05-11_03:57:34-orch:cephadm-wip-adk-testing-2022-05-10-2040-distro-basic-smithi/6830310 --name adking-2022-05-11_03:57:34-orch:cephadm-wip-adk-testing-2022-05-10-2040-distro-basic-smithi --description orch:cephadm/smoke-roleless/{0-distro/rhel_8.5_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} -- /home/teuthworker/archive/adking-2022-05-11_03:57:34-orch:cephadm-wip-adk-testing-2022-05-10-2040-distro-basic-smithi/6830310/orig.config.yaml 2022-05-11T07:45:40.995 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2022-05-11_03:57:34-orch:cephadm-wip-adk-testing-2022-05-10-2040-distro-basic-smithi/6830310 2022-05-11T07:45:40.996 INFO:teuthology.dispatcher.supervisor:Job PID: 9776 2022-05-11T07:45:40.996 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-05-11T14:16:05.206 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-05-11T14:16:05.550 INFO:teuthology.kill:Killing Pids: {9776} 2022-05-11T14:16:06.809 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-05-11T14:16:06.810 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-05-11T14:16:06.810 INFO:teuthology.misc:Compressing logs... 2022-05-11T14:16:06.811 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2022-05-11T14:16:07.011 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2022-05-11T14:16:07.075 INFO:teuthology.orchestra.run.smithi131.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-05-11T14:16:07.237 INFO:teuthology.orchestra.run.smithi139.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-05-11T14:16:07.270 INFO:teuthology.orchestra.run.smithi131.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-05-11T14:16:07.271 INFO:teuthology.orchestra.run.smithi131.stderr:tar: Error is not recoverable: exiting now 2022-05-11T14:16:07.306 INFO:teuthology.orchestra.run.smithi139.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-05-11T14:16:07.307 INFO:teuthology.orchestra.run.smithi139.stderr:tar: Error is not recoverable: exiting now 2022-05-11T14:16:07.308 INFO:teuthology.misc:Compressing logs... 2022-05-11T14:16:07.511 INFO:teuthology.misc:Compressing logs... 2022-05-11T14:16:09.464 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/0b6367b8-d11b-11ec-8c39-001a4aab830c/ceph-mon.smithi139.log: file size changed while zipping 2022-05-11T14:16:10.935 INFO:teuthology.orchestra.run.smithi131.stderr:gzip: /var/log/ceph/0b6367b8-d11b-11ec-8c39-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-05-11T14:16:13.001 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/0b6367b8-d11b-11ec-8c39-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-05-11T14:16:14.216 INFO:teuthology.orchestra.run.smithi131.stderr:gzip: /var/log/ceph/0b6367b8-d11b-11ec-8c39-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-05-11T14:16:15.985 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/0b6367b8-d11b-11ec-8c39-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-05-11T14:16:17.878 INFO:teuthology.orchestra.run.smithi131.stderr:gzip: /var/log/ceph/0b6367b8-d11b-11ec-8c39-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-05-11T14:16:19.512 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/0b6367b8-d11b-11ec-8c39-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-05-11T14:16:20.845 INFO:teuthology.orchestra.run.smithi131.stderr:gzip: /var/log/ceph/0b6367b8-d11b-11ec-8c39-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-05-11T14:16:23.271 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/0b6367b8-d11b-11ec-8c39-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-05-11T14:16:25.673 INFO:teuthology.orchestra.run.smithi131.stderr:gzip: /var/log/ceph/0b6367b8-d11b-11ec-8c39-001a4aab830c/ceph-mon.smithi131.log: file size changed while zipping 2022-05-11T14:16:26.975 INFO:teuthology.orchestra.run.smithi131.stderr:gzip: /var/log/ceph/0b6367b8-d11b-11ec-8c39-001a4aab830c/ceph-mgr.smithi131.tdxgkt.log: file size changed while zipping 2022-05-11T14:58:48.376 INFO:teuthology.orchestra.run.smithi131.stderr:tar: ./0b6367b8-d11b-11ec-8c39-001a4aab830c/ceph-osd.4.log.gz: file changed as we read it 2022-05-11T15:19:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:tar: ./0b6367b8-d11b-11ec-8c39-001a4aab830c: file changed as we read it 2022-05-11T17:19:06.680 INFO:teuthology.orchestra.run.smithi139.stderr:tar: ./0b6367b8-d11b-11ec-8c39-001a4aab830c: file changed as we read it 2022-05-11T17:19:07.175 INFO:teuthology.kill:No teuthology processes running 2022-05-11T17:19:08.564 INFO:teuthology.kill:Nuking machines: ['smithi131', 'smithi139'] 2022-05-11T17:19:09.799 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b018fa018d3f885cbc018f839b42536477653f84/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-05-11T17:19:09.800 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-05-11T17:19:09.800 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b018fa018d3f885cbc018f839b42536477653f84/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated 2022-05-11T17:19:09.801 INFO:teuthology.kill: "class": algorithms.Blowfish, 2022-05-11T17:19:09.801 INFO:teuthology.kill:2022-05-11 17:19:09,799.799 INFO:teuthology.nuke:targets: 2022-05-11T17:19:09.801 INFO:teuthology.kill: smithi131.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCl1CMLE05Fj9+YQUzrvWN4td+687dvzH4lIQIKrqwbbelg/xYh8V5UcsTUPDecFs7izAHp6ZQEmtkh9A+dO2iDr/20eULTRH0MFA8XOOG9QXOghVcv+6lkofklR+wO6ifMAqDSwonf0pYGm3FOzAp9LZ2YUOWIeG2xqAqfuBt+hEuQockA6Yg5zV4ZQP+6D8RSPqHCl4nDJI7w1mLNvjecrREEodl6/WX0q7pw45LpvFMISwMwFlrAS2df6P5G/PO7J4v+zUlsbfHPg5Xh94xgzWEIWAwiAIXs68s5Rd+YZiRhCtvc2m4sda2GRovCnjPWLTeQ7FsCda9UctywFgKzqZT4uBxEg4iRvDYgsV/2hhExLL+oovH4J2/F84HJsj/UDAc4PAN4tcJAg9vZO/5gWICinbRdSqwtMlG/wuakQNWH69xCyYzQSyS3I/1/RyiGozsd8UG72ZsmT2t4CI7Xvzn3x4/Ts+W4Exb9MUgnqAECQs2BwDkC8tBjeYlFnCs= 2022-05-11T17:19:09.801 INFO:teuthology.kill: smithi139.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsdoUzO8EN8BwVOl1Ln6Z3gIMrxPFmPN1AQhMvqsnIxYRVUNAVC5gMXh+KYc05o9hEJE5Luo21TDn8Nq+yaunLlJsA9pJ7JdqvetzFpw/XzghTDsmXArFG75Ipn2dnqlDpwbCvnSVCbDSNxZs4pxtFNF+p4aS9QKu27dJP9ogdTiofyCsuoahVj/Vej3Qm1jqzbpsRum+XdRvfq1D8MGD0QRCkWPGyIyjWXDmMFxBTLXF+cVyWHRjgUWr19VQzi9mV5I1LYWs5MkMrplorZbQHT9z0XVVT+B/au/rjK4nI7t235PlrnQbcj8hqNhuQ019McYj9aw8OWjEFsMX+RaQDl+/vv91uA1D1GxkgGCx+1a8sHe+XB7zZZDmHuNYm9eItP/xj9KuFjB8jzKmx/LITmpwE9/EL2Tb5Qj0W9+l9UpK68af5TGNupnDVxSRFrzdlI2MjsXAWKHslaAlSgsiykiMdXbC8BQy4T2eb/Zrfvocb3PKg89SmMV2RYCm+11k= 2022-05-11T17:19:09.830 INFO:teuthology.kill:2022-05-11 17:19:09,830.830 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-05-11T17:19:09.831 INFO:teuthology.kill:2022-05-11 17:19:09,831.831 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-05-11T17:19:09.902 INFO:teuthology.kill:2022-05-11 17:19:09,902.902 INFO:teuthology.orchestra.console:Power off smithi131 2022-05-11T17:19:09.914 INFO:teuthology.kill:2022-05-11 17:19:09,914.914 INFO:teuthology.orchestra.console:Power off smithi139 2022-05-11T17:19:18.272 INFO:teuthology.kill:2022-05-11 17:19:18,271.271 INFO:teuthology.orchestra.console:Power off for smithi139 completed 2022-05-11T17:19:18.429 INFO:teuthology.kill:2022-05-11 17:19:18,429.429 INFO:teuthology.lock.ops:unlocked: smithi139.front.sepia.ceph.com 2022-05-11T17:19:22.377 INFO:teuthology.kill:2022-05-11 17:19:22,377.377 INFO:teuthology.orchestra.console:Power off for smithi131 completed 2022-05-11T17:19:22.517 INFO:teuthology.kill:2022-05-11 17:19:22,517.517 INFO:teuthology.lock.ops:unlocked: smithi131.front.sepia.ceph.com 2022-05-11T17:21:22.803 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15