2022-01-26T05:52:24.200 INFO:root:teuthology version: 1.1.0-7c0cb867 2022-01-26T05:52:24.318 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging 2022-01-26T05:52:24.319 INFO:teuthology.lock.ops:Updating [ubuntu@smithi146.front.sepia.ceph.com]: reset os type and version on server 2022-01-26T05:52:24.319 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2022-01-26T05:52:24.339 INFO:teuthology.lock.ops:Node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging is complete 2022-01-26T05:52:24.339 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging 2022-01-26T05:52:24.339 INFO:teuthology.lock.ops:Updating [ubuntu@smithi185.front.sepia.ceph.com]: reset os type and version on server 2022-01-26T05:52:24.340 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2022-01-26T05:52:24.360 INFO:teuthology.lock.ops:Node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging is complete 2022-01-26T05:52:24.553 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of rhel 8.4 2022-01-26T05:52:24.577 INFO:teuthology.provision.fog.smithi185:Scheduling deploy of rhel 8.4 2022-01-26T05:52:24.941 INFO:teuthology.orchestra.console:Power off smithi146 2022-01-26T05:52:24.979 INFO:teuthology.orchestra.console:Power off smithi185 2022-01-26T05:52:33.312 INFO:teuthology.orchestra.console:Power off for smithi146 completed 2022-01-26T05:52:33.346 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2022-01-26T05:52:33.414 INFO:teuthology.orchestra.console:Power on smithi146 2022-01-26T05:52:33.448 INFO:teuthology.orchestra.console:Power on smithi185 2022-01-26T05:52:41.801 INFO:teuthology.orchestra.console:Power on for smithi146 completed 2022-01-26T05:52:41.809 INFO:teuthology.orchestra.console:Power on for smithi185 completed 2022-01-26T05:52:41.902 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish 2022-01-26T05:52:41.910 INFO:teuthology.provision.fog.smithi185:Waiting for deploy to finish 2022-01-26T05:57:03.577 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-26T05:57:24.027 INFO:teuthology.provision.fog.smithi146:Node is ready 2022-01-26T05:57:24.047 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com 2022-01-26T05:57:24.104 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146 2022-01-26T05:57:24.459 INFO:teuthology.provision.fog.smithi146:Deploy complete! 2022-01-26T05:57:37.803 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-26T05:57:38.003 INFO:teuthology.provision.fog.smithi185:Node is ready 2022-01-26T05:57:38.026 INFO:teuthology.orchestra.run.smithi185.stdout:smithi185.front.sepia.ceph.com 2022-01-26T05:57:38.081 INFO:teuthology.orchestra.run.smithi185.stdout:172.21.15.185 smithi185.front.sepia.ceph.com smithi185 2022-01-26T05:57:38.408 INFO:teuthology.provision.fog.smithi185:Deploy complete! 2022-01-26T05:57:38.566 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com 2022-01-26T05:57:38.567 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-26T05:57:38.590 INFO:teuthology.lock.ops:Checking smithi185.front.sepia.ceph.com 2022-01-26T05:57:38.590 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-26T05:57:38.611 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server 2022-01-26T05:57:38.612 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2022-01-26T05:57:38.844 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2022-01-26T05:57:38.899 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="Red Hat Enterprise Linux" 2022-01-26T05:57:38.900 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8.4 (Ootpa)" 2022-01-26T05:57:38.900 INFO:teuthology.orchestra.run.smithi146.stdout:ID="rhel" 2022-01-26T05:57:38.900 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="fedora" 2022-01-26T05:57:38.900 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8.4" 2022-01-26T05:57:38.901 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8" 2022-01-26T05:57:38.901 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-26T05:57:38.901 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2022-01-26T05:57:38.901 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-26T05:57:38.902 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://www.redhat.com/" 2022-01-26T05:57:38.902 INFO:teuthology.orchestra.run.smithi146.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-26T05:57:38.902 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-26T05:57:38.902 INFO:teuthology.orchestra.run.smithi146.stdout: 2022-01-26T05:57:38.902 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-26T05:57:38.903 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-26T05:57:38.903 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-26T05:57:38.903 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-26T05:57:38.905 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2022-01-26T05:57:38.922 INFO:teuthology.lock.ops:Updating [smithi185.front.sepia.ceph.com]: set os type and version on server 2022-01-26T05:57:38.923 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com' 2022-01-26T05:57:39.151 INFO:teuthology.orchestra.run.smithi185.stdout:x86_64 2022-01-26T05:57:39.204 INFO:teuthology.orchestra.run.smithi185.stdout:NAME="Red Hat Enterprise Linux" 2022-01-26T05:57:39.205 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION="8.4 (Ootpa)" 2022-01-26T05:57:39.205 INFO:teuthology.orchestra.run.smithi185.stdout:ID="rhel" 2022-01-26T05:57:39.205 INFO:teuthology.orchestra.run.smithi185.stdout:ID_LIKE="fedora" 2022-01-26T05:57:39.205 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_ID="8.4" 2022-01-26T05:57:39.206 INFO:teuthology.orchestra.run.smithi185.stdout:PLATFORM_ID="platform:el8" 2022-01-26T05:57:39.206 INFO:teuthology.orchestra.run.smithi185.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-26T05:57:39.206 INFO:teuthology.orchestra.run.smithi185.stdout:ANSI_COLOR="0;31" 2022-01-26T05:57:39.206 INFO:teuthology.orchestra.run.smithi185.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-26T05:57:39.207 INFO:teuthology.orchestra.run.smithi185.stdout:HOME_URL="https://www.redhat.com/" 2022-01-26T05:57:39.207 INFO:teuthology.orchestra.run.smithi185.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-26T05:57:39.207 INFO:teuthology.orchestra.run.smithi185.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-26T05:57:39.207 INFO:teuthology.orchestra.run.smithi185.stdout: 2022-01-26T05:57:39.208 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-26T05:57:39.208 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-26T05:57:39.208 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-26T05:57:39.208 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-26T05:57:39.209 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2022-01-26T05:57:39.297 INFO:teuthology.dispatcher.supervisor:Running job 6641112 2022-01-26T05:57:39.298 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7c0cb8672986d9dbe53078a123af65593653ef7a/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2022-01-26_05:14:25-rados:cephadm-wip-adk2-testing-2022-01-25-2148-pacific-distro-basic-smithi/6641112 --name adking-2022-01-26_05:14:25-rados:cephadm-wip-adk2-testing-2022-01-25-2148-pacific-distro-basic-smithi --description rados:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} -- /home/teuthworker/archive/adking-2022-01-26_05:14:25-rados:cephadm-wip-adk2-testing-2022-01-25-2148-pacific-distro-basic-smithi/6641112/orig.config.yaml 2022-01-26T05:57:39.300 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2022-01-26_05:14:25-rados:cephadm-wip-adk2-testing-2022-01-25-2148-pacific-distro-basic-smithi/6641112 2022-01-26T05:57:39.301 INFO:teuthology.dispatcher.supervisor:Job PID: 27106 2022-01-26T05:57:39.302 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-26T12:28:09.135 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-26T12:28:09.340 INFO:teuthology.kill:Killing Pids: {27106} 2022-01-26T12:28:10.476 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-26T12:28:10.476 INFO:teuthology.task.internal:roles: ubuntu@smithi185.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-26T12:28:10.477 INFO:teuthology.misc:Compressing logs... 2022-01-26T12:28:10.477 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2022-01-26T12:28:10.667 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com' 2022-01-26T12:28:10.713 INFO:teuthology.orchestra.run.smithi146.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-26T12:28:10.897 INFO:teuthology.orchestra.run.smithi185.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-26T12:28:10.928 INFO:teuthology.orchestra.run.smithi146.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-26T12:28:10.929 INFO:teuthology.orchestra.run.smithi146.stderr:tar: Error is not recoverable: exiting now 2022-01-26T12:28:10.956 INFO:teuthology.orchestra.run.smithi185.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-26T12:28:10.957 INFO:teuthology.orchestra.run.smithi185.stderr:tar: Error is not recoverable: exiting now 2022-01-26T12:28:10.958 INFO:teuthology.misc:Compressing logs... 2022-01-26T12:28:11.180 INFO:teuthology.misc:Compressing logs... 2022-01-26T12:28:17.953 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/598380ae-7e6e-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-26T12:28:19.971 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/598380ae-7e6e-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-26T12:28:23.370 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/598380ae-7e6e-11ec-8c35-001a4aab830c/ceph-mon.smithi185.log: file size changed while zipping 2022-01-26T12:28:27.137 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/598380ae-7e6e-11ec-8c35-001a4aab830c/ceph-mon.smithi146.log: file size changed while zipping 2022-01-26T12:28:29.786 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/598380ae-7e6e-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-26T12:28:34.059 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/598380ae-7e6e-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-26T12:28:38.466 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/598380ae-7e6e-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-26T12:28:41.645 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/598380ae-7e6e-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-26T12:28:45.249 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/598380ae-7e6e-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-26T12:28:50.711 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/598380ae-7e6e-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-26T12:28:54.602 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/598380ae-7e6e-11ec-8c35-001a4aab830c/ceph-mgr.smithi146.nvfhjm.log: file size changed while zipping 2022-01-26T12:29:10.723 INFO:teuthology.kill:No teuthology processes running 2022-01-26T12:29:11.856 INFO:teuthology.kill:Nuking machines: ['smithi146', 'smithi185'] 2022-01-26T12:29:12.798 INFO:teuthology.kill:2022-01-26 12:29:12,798.798 INFO:teuthology.nuke:targets: 2022-01-26T12:29:12.799 INFO:teuthology.kill: smithi146.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDW7s60U+YryE+7Juklq9Yac6MC6uemlobMLzjOME2zpbS8MRhNLgtE2mnkG87s0Vc4FO9trLhoekRtDDw+cI2YLLy5m9oQsfev8fSFiMVW7dVIBNYcyL5IESARS2vBAmTpA9dCWsdIxUWGk1AWAZFqrGKJLMvrnOAxhy+NRAP9u/tszm5jCwGdLmV2zoCD0hBsvABJwKG2ymifFuwRaNk1Zo12QO1cHfvYmNYTGbJr9lt6/3mSJi6EwPQgyWCQ+8wNsD4iCYqFwtcl6SHStOGJYH8rfsmwVFj5ITrWhPUcecLX9d2TcHS3bIIYhKMKwKO3sp+XtOq2OnU6simT4ORVFxJwfhuQj+xwHLKt6AhZ63pFTVhCmlcluSy5PiYnMp/7+iEMwZYqP6g5trcQ8Oj8YCIT/5zMMD9NOD2AVh+4x9L/E20wVi6Jop5RQnLbt7pULCl3zrMcxZVMAVcf2ara5zoPSwQLZZcRCyKx1CwePhOl4l7DupUtlA5tWRJluTU= 2022-01-26T12:29:12.799 INFO:teuthology.kill: smithi185.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNk4xoO5ebdYdaV1UK3eUCg6PBdJYbO/dydG0VBlOTsgYF76RdvLBogYZK8VeJbvpTrxKHiBeDZO39q081A1duAde/ofRBAsRpfDBlm2PzDZe2GeZ4/MpcmJbW9aYIJW9LAKwXjduYpWbxnyv1ngGd7X4UYH0US+g7YqGXhOgYCwVaN5prtSJkUhHy85bqQUKEsitFPOMuAIdShr3cA5P/iuNyQVqfdY4MP1DB731dzQCPxV7NBNTaZ80xYcTjt9u4hDIlNaimgptaEbZYiGnyFpPlI4+52Q11wjuJMmZfnhjitAcufIFRiHW1rxbIfIqzjEl4gcLXOoQxOGpHu/OeruTGc6KCWTdmcyD/CrwcpNscjScgwW/o9Md4c9rs6BoC+6hmiLl1S7kmIs2FxGm4QlZIokaekKW51WE1ecZ5ZmK3RJu/VnrcRbSXJuk0QvM3h8UbUJn9hORjK6WUyRBQcpkMPNIKU27mtNMh4yJ4K26E6p0QeXH2lwvCyJMLThE= 2022-01-26T12:29:12.825 INFO:teuthology.kill:2022-01-26 12:29:12,824.824 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-26T12:29:12.828 INFO:teuthology.kill:2022-01-26 12:29:12,828.828 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-26T12:29:12.911 INFO:teuthology.kill:2022-01-26 12:29:12,910.910 INFO:teuthology.orchestra.console:Power off smithi146 2022-01-26T12:29:12.925 INFO:teuthology.kill:2022-01-26 12:29:12,925.925 INFO:teuthology.orchestra.console:Power off smithi185 2022-01-26T12:29:21.282 INFO:teuthology.kill:2022-01-26 12:29:21,282.282 INFO:teuthology.orchestra.console:Power off for smithi146 completed 2022-01-26T12:29:21.288 INFO:teuthology.kill:2022-01-26 12:29:21,288.288 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2022-01-26T12:29:21.429 INFO:teuthology.kill:2022-01-26 12:29:21,428.428 INFO:teuthology.lock.ops:unlocked: smithi146.front.sepia.ceph.com 2022-01-26T12:29:21.431 INFO:teuthology.kill:2022-01-26 12:29:21,431.431 INFO:teuthology.lock.ops:unlocked: smithi185.front.sepia.ceph.com 2022-01-26T12:31:21.740 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-26T12:31:21.835 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi146 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-26_12:17:03-powercycle-pacific-distro-default-smithi/6641363 2022-01-26T12:31:21.835 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi185 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-26_12:17:03-powercycle-pacific-distro-default-smithi/6641362