2022-01-23T00:44:25.236 INFO:root:teuthology version: 1.1.0-75f1bcd2 2022-01-23T00:44:25.377 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi042.front.sepia.ceph.com' reimaging 2022-01-23T00:44:25.378 INFO:teuthology.lock.ops:Updating [ubuntu@smithi042.front.sepia.ceph.com]: reset os type and version on server 2022-01-23T00:44:25.380 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2022-01-23T00:44:25.401 INFO:teuthology.lock.ops:Node 'ubuntu@smithi042.front.sepia.ceph.com' reimaging is complete 2022-01-23T00:44:25.402 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging 2022-01-23T00:44:25.402 INFO:teuthology.lock.ops:Updating [ubuntu@smithi185.front.sepia.ceph.com]: reset os type and version on server 2022-01-23T00:44:25.402 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2022-01-23T00:44:25.422 INFO:teuthology.lock.ops:Node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging is complete 2022-01-23T00:44:25.606 INFO:teuthology.provision.fog.smithi042:Scheduling deploy of rhel 8.4 2022-01-23T00:44:25.620 INFO:teuthology.provision.fog.smithi185:Scheduling deploy of rhel 8.4 2022-01-23T00:44:25.941 INFO:teuthology.orchestra.console:Power off smithi185 2022-01-23T00:44:25.959 INFO:teuthology.orchestra.console:Power off smithi042 2022-01-23T00:44:34.318 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2022-01-23T00:44:34.323 INFO:teuthology.orchestra.console:Power off for smithi042 completed 2022-01-23T00:44:34.421 INFO:teuthology.orchestra.console:Power on smithi185 2022-01-23T00:44:34.428 INFO:teuthology.orchestra.console:Power on smithi042 2022-01-23T00:44:42.787 INFO:teuthology.orchestra.console:Power on for smithi185 completed 2022-01-23T00:44:42.792 INFO:teuthology.orchestra.console:Power on for smithi042 completed 2022-01-23T00:44:42.889 INFO:teuthology.provision.fog.smithi185:Waiting for deploy to finish 2022-01-23T00:44:42.892 INFO:teuthology.provision.fog.smithi042:Waiting for deploy to finish 2022-01-23T00:48:50.463 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-23T00:48:50.691 INFO:teuthology.provision.fog.smithi042:Node is ready 2022-01-23T00:48:50.707 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042.front.sepia.ceph.com 2022-01-23T00:48:50.781 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42 smithi042.front.sepia.ceph.com smithi042 2022-01-23T00:48:51.125 INFO:teuthology.provision.fog.smithi042:Deploy complete! 2022-01-23T00:49:04.207 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-23T00:49:24.624 INFO:teuthology.provision.fog.smithi185:Node is ready 2022-01-23T00:49:24.781 INFO:teuthology.orchestra.run.smithi185.stdout:smithi185.front.sepia.ceph.com 2022-01-23T00:49:24.838 INFO:teuthology.orchestra.run.smithi185.stdout:172.21.15.185 smithi185.front.sepia.ceph.com smithi185 2022-01-23T00:49:25.185 INFO:teuthology.provision.fog.smithi185:Deploy complete! 2022-01-23T00:49:25.348 INFO:teuthology.lock.ops:Checking smithi042.front.sepia.ceph.com 2022-01-23T00:49:25.349 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-23T00:49:25.369 INFO:teuthology.lock.ops:Checking smithi185.front.sepia.ceph.com 2022-01-23T00:49:25.369 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-23T00:49:25.388 INFO:teuthology.lock.ops:Updating [smithi042.front.sepia.ceph.com]: set os type and version on server 2022-01-23T00:49:25.389 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2022-01-23T00:49:25.631 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2022-01-23T00:49:25.687 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="Red Hat Enterprise Linux" 2022-01-23T00:49:25.688 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="8.4 (Ootpa)" 2022-01-23T00:49:25.688 INFO:teuthology.orchestra.run.smithi042.stdout:ID="rhel" 2022-01-23T00:49:25.688 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="fedora" 2022-01-23T00:49:25.688 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="8.4" 2022-01-23T00:49:25.689 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el8" 2022-01-23T00:49:25.689 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-23T00:49:25.689 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2022-01-23T00:49:25.689 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-23T00:49:25.690 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://www.redhat.com/" 2022-01-23T00:49:25.690 INFO:teuthology.orchestra.run.smithi042.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-23T00:49:25.690 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-23T00:49:25.690 INFO:teuthology.orchestra.run.smithi042.stdout: 2022-01-23T00:49:25.691 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-23T00:49:25.691 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-23T00:49:25.691 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-23T00:49:25.691 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-23T00:49:25.693 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2022-01-23T00:49:25.711 INFO:teuthology.lock.ops:Updating [smithi185.front.sepia.ceph.com]: set os type and version on server 2022-01-23T00:49:25.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com' 2022-01-23T00:49:25.945 INFO:teuthology.orchestra.run.smithi185.stdout:x86_64 2022-01-23T00:49:25.999 INFO:teuthology.orchestra.run.smithi185.stdout:NAME="Red Hat Enterprise Linux" 2022-01-23T00:49:26.000 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION="8.4 (Ootpa)" 2022-01-23T00:49:26.000 INFO:teuthology.orchestra.run.smithi185.stdout:ID="rhel" 2022-01-23T00:49:26.000 INFO:teuthology.orchestra.run.smithi185.stdout:ID_LIKE="fedora" 2022-01-23T00:49:26.000 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_ID="8.4" 2022-01-23T00:49:26.001 INFO:teuthology.orchestra.run.smithi185.stdout:PLATFORM_ID="platform:el8" 2022-01-23T00:49:26.001 INFO:teuthology.orchestra.run.smithi185.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-23T00:49:26.001 INFO:teuthology.orchestra.run.smithi185.stdout:ANSI_COLOR="0;31" 2022-01-23T00:49:26.001 INFO:teuthology.orchestra.run.smithi185.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-23T00:49:26.002 INFO:teuthology.orchestra.run.smithi185.stdout:HOME_URL="https://www.redhat.com/" 2022-01-23T00:49:26.002 INFO:teuthology.orchestra.run.smithi185.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-23T00:49:26.002 INFO:teuthology.orchestra.run.smithi185.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-23T00:49:26.002 INFO:teuthology.orchestra.run.smithi185.stdout: 2022-01-23T00:49:26.003 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-23T00:49:26.003 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-23T00:49:26.003 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-23T00:49:26.003 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-23T00:49:26.004 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2022-01-23T00:49:26.107 INFO:teuthology.dispatcher.supervisor:Running job 6635439 2022-01-23T00:49:26.108 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_75f1bcd237a8b6022305cd53ba87e106d45aa665/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-22_17:47:39-rados-wip-yuriw-quincy-1.22.22-distro-default-smithi/6635439 --name yuriw-2022-01-22_17:47:39-rados-wip-yuriw-quincy-1.22.22-distro-default-smithi --description rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} -- /home/teuthworker/archive/yuriw-2022-01-22_17:47:39-rados-wip-yuriw-quincy-1.22.22-distro-default-smithi/6635439/orig.config.yaml 2022-01-23T00:49:26.113 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-22_17:47:39-rados-wip-yuriw-quincy-1.22.22-distro-default-smithi/6635439 2022-01-23T00:49:26.114 INFO:teuthology.dispatcher.supervisor:Job PID: 27371 2022-01-23T00:49:26.115 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-23T07:19:56.335 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-23T07:19:56.514 INFO:teuthology.kill:Killing Pids: {27371} 2022-01-23T07:19:57.788 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-23T07:19:57.789 INFO:teuthology.task.internal:roles: ubuntu@smithi185.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-23T07:19:57.789 INFO:teuthology.misc:Compressing logs... 2022-01-23T07:19:57.790 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2022-01-23T07:19:57.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com' 2022-01-23T07:19:58.028 INFO:teuthology.orchestra.run.smithi042.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-23T07:19:58.216 INFO:teuthology.orchestra.run.smithi185.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-23T07:19:58.263 INFO:teuthology.orchestra.run.smithi042.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-23T07:19:58.263 INFO:teuthology.orchestra.run.smithi042.stderr:tar: Error is not recoverable: exiting now 2022-01-23T07:19:58.378 INFO:teuthology.orchestra.run.smithi185.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-23T07:19:58.379 INFO:teuthology.orchestra.run.smithi185.stderr:tar: Error is not recoverable: exiting now 2022-01-23T07:19:58.380 INFO:teuthology.misc:Compressing logs... 2022-01-23T07:19:58.653 INFO:teuthology.misc:Compressing logs... 2022-01-23T07:20:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/dc4f8eda-7be7-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-23T07:20:07.986 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/dc4f8eda-7be7-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-23T07:20:11.412 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/dc4f8eda-7be7-11ec-8c35-001a4aab830c/ceph-mon.smithi185.log: file size changed while zipping 2022-01-23T07:20:14.107 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/dc4f8eda-7be7-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-23T07:20:18.352 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/dc4f8eda-7be7-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-23T07:20:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/dc4f8eda-7be7-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-23T07:20:27.628 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/dc4f8eda-7be7-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-23T07:20:33.031 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/dc4f8eda-7be7-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-23T07:20:34.887 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/dc4f8eda-7be7-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-23T07:20:42.827 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/dc4f8eda-7be7-11ec-8c35-001a4aab830c/ceph-mon.smithi042.log: file size changed while zipping 2022-01-23T07:20:44.635 INFO:teuthology.orchestra.run.smithi042.stderr:gzip: /var/log/ceph/dc4f8eda-7be7-11ec-8c35-001a4aab830c/ceph-mgr.smithi042.dgknvy.log: file size changed while zipping 2022-01-23T07:21:01.976 INFO:teuthology.kill:No teuthology processes running 2022-01-23T07:21:03.205 INFO:teuthology.kill:Nuking machines: ['smithi042', 'smithi185'] 2022-01-23T07:21:04.215 INFO:teuthology.kill:2022-01-23 07:21:04,215.215 INFO:teuthology.nuke:targets: 2022-01-23T07:21:04.216 INFO:teuthology.kill: smithi042.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5PLqT9CcztL4Ub1/Ih4fRUcwtM7N30HlLZNcT0KdKZR1dMC3yQhE/y8mEeSRGOGf21yfI3IyjEp6mY15NvUE8DEA21cb82WhKOCameqkjFc/GRh+W4amPZSA2U23XBOngSI+jQz9WwGN6QtG3F1pC0lKYo3eQa+3I+J75rnzaMDCtAs7cK2BHISQjq/N31a9eGmykYBuzf1XCD7pGOoYG2pU8Zqz6MxKG41NDMuurl2uxpzJriM3tiade9Cb4S+C9XZ9NC329OFSMuxcTo+PsLx1LIc0bO6KTdRwoGK75FaPzLHlOhQJI5qEIESnmA2YzlK/W3iByBN1j1WUlQJhj/Fg87D8nk/8o+HQZAUyuQ9rMPMqSlddBwo+kRF3WrJzd1RSXlHtYdR9ZNwvmoSF//HtA8/2BxqcX9QvjOdfT2HnI72OS2GH2mVEUXy3vrpqOwDDzVD/czlY1TPexkFCiuTeOtMkyGmc4jFfSIjmz8ZYDO1SkPSBEHLOwb4ohWh8= 2022-01-23T07:21:04.216 INFO:teuthology.kill: smithi185.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI0nUGVz8yPloSrt11biUBIISvCvBo545dhUZ709zlERhMidl0U04FG9JnP0+3UocD6MG5Hd1T7bHAo2jVBmpoMpGQez1rd8kaxcr4B0nN/Eourez7acm2K/KNBl1fgMlxLDQI+BBGL0C0aLqcW/YP/R7YMoFOMan/CIihmeV8nuVVOXIxIbALU09NMa6gbvWwaB9FOwlbCOZ7PmcUnqdJbHtuDOFKMojM2/LwEywzZxBp3//Vj0C8bCRTYuFPAVX+QBJqLYloffXOzIAJu8bdyS5HrxdbHZF4SrTmKUgmh20LhVjOnU2Wck1KMPrMsKDHwZiY6nXoDOuZV4xOPvU1DBcjhpPPzxYSRMhJePAx9f0LV4VoWtvhfNYSeGbAFsSVxmhkZ4AhFcPHEkecTUqsql7DM7Hp1scfpyGwg25mpqBmGwNh3OUDd5TZXVypX/MY5HFAvOzO/i0hdTEeGdn5z9nk5n4lup4KKt4a8cMTtGSTqK7Da/r51IxoqEoRjI8= 2022-01-23T07:21:04.241 INFO:teuthology.kill:2022-01-23 07:21:04,241.241 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-23T07:21:04.245 INFO:teuthology.kill:2022-01-23 07:21:04,245.245 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-23T07:21:04.323 INFO:teuthology.kill:2022-01-23 07:21:04,322.322 INFO:teuthology.orchestra.console:Power off smithi185 2022-01-23T07:21:04.337 INFO:teuthology.kill:2022-01-23 07:21:04,337.337 INFO:teuthology.orchestra.console:Power off smithi042 2022-01-23T07:21:12.696 INFO:teuthology.kill:2022-01-23 07:21:12,696.696 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2022-01-23T07:21:12.850 INFO:teuthology.kill:2022-01-23 07:21:12,849.849 INFO:teuthology.lock.ops:unlocked: smithi185.front.sepia.ceph.com 2022-01-23T07:21:16.817 INFO:teuthology.kill:2022-01-23 07:21:16,816.816 INFO:teuthology.orchestra.console:Power off for smithi042 completed 2022-01-23T07:21:16.965 INFO:teuthology.kill:2022-01-23 07:21:16,965.965 INFO:teuthology.lock.ops:unlocked: smithi042.front.sepia.ceph.com 2022-01-23T07:23:17.331 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-23T07:23:17.438 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi042 but it was locked by another job: /home/teuthworker/archive/benhanokh-2022-01-22_16:49:51-rados-WIP_GBH_NCB_new_alloc_map_A1-distro-basic-smithi/6634667 2022-01-23T07:23:17.438 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi185 but it was locked by another job: /home/teuthworker/archive/benhanokh-2022-01-22_16:49:51-rados-WIP_GBH_NCB_new_alloc_map_A1-distro-basic-smithi/6634661