2022-02-11T17:16:14.633 INFO:root:teuthology version: 1.1.0-7cebb3f2 2022-02-11T17:16:14.756 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi124.front.sepia.ceph.com' reimaging 2022-02-11T17:16:14.759 INFO:teuthology.lock.ops:Updating [ubuntu@smithi124.front.sepia.ceph.com]: reset os type and version on server 2022-02-11T17:16:14.759 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2022-02-11T17:16:14.782 INFO:teuthology.lock.ops:Node 'ubuntu@smithi124.front.sepia.ceph.com' reimaging is complete 2022-02-11T17:16:14.783 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi183.front.sepia.ceph.com' reimaging 2022-02-11T17:16:14.783 INFO:teuthology.lock.ops:Updating [ubuntu@smithi183.front.sepia.ceph.com]: reset os type and version on server 2022-02-11T17:16:14.784 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2022-02-11T17:16:14.805 INFO:teuthology.lock.ops:Node 'ubuntu@smithi183.front.sepia.ceph.com' reimaging is complete 2022-02-11T17:16:14.995 INFO:teuthology.provision.fog.smithi124:Scheduling deploy of rhel 8.4 2022-02-11T17:16:14.998 INFO:teuthology.provision.fog.smithi183:Scheduling deploy of rhel 8.4 2022-02-11T17:16:15.310 INFO:teuthology.orchestra.console:Power off smithi124 2022-02-11T17:16:15.329 INFO:teuthology.orchestra.console:Power off smithi183 2022-02-11T17:16:23.684 INFO:teuthology.orchestra.console:Power off for smithi124 completed 2022-02-11T17:16:23.688 INFO:teuthology.orchestra.console:Power off for smithi183 completed 2022-02-11T17:16:23.786 INFO:teuthology.orchestra.console:Power on smithi124 2022-02-11T17:16:23.795 INFO:teuthology.orchestra.console:Power on smithi183 2022-02-11T17:16:32.152 INFO:teuthology.orchestra.console:Power on for smithi124 completed 2022-02-11T17:16:32.212 INFO:teuthology.orchestra.console:Power on for smithi183 completed 2022-02-11T17:16:32.254 INFO:teuthology.provision.fog.smithi124:Waiting for deploy to finish 2022-02-11T17:16:32.313 INFO:teuthology.provision.fog.smithi183:Waiting for deploy to finish 2022-02-11T17:20:44.725 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-11T17:20:59.825 INFO:teuthology.provision.fog.smithi124:Node is ready 2022-02-11T17:20:59.844 INFO:teuthology.orchestra.run.smithi124.stdout:smithi204 2022-02-11T17:20:59.900 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.15.204 smithi204.front.sepia.ceph.com smithi204 2022-02-11T17:21:00.231 INFO:teuthology.provision.fog.smithi124:Deploy complete! 2022-02-11T17:21:10.337 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-11T17:21:10.537 INFO:teuthology.provision.fog.smithi183:Node is ready 2022-02-11T17:21:10.552 INFO:teuthology.orchestra.run.smithi183.stdout:smithi183.front.sepia.ceph.com 2022-02-11T17:21:10.607 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.15.183 smithi183.front.sepia.ceph.com smithi183 2022-02-11T17:21:10.945 INFO:teuthology.provision.fog.smithi183:Deploy complete! 2022-02-11T17:21:11.090 INFO:teuthology.lock.ops:Checking smithi124.front.sepia.ceph.com 2022-02-11T17:21:11.091 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-11T17:21:11.112 INFO:teuthology.lock.ops:Checking smithi183.front.sepia.ceph.com 2022-02-11T17:21:11.113 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-11T17:21:11.132 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: set os type and version on server 2022-02-11T17:21:11.133 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2022-02-11T17:21:11.368 INFO:teuthology.orchestra.run.smithi124.stdout:x86_64 2022-02-11T17:21:11.423 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="Red Hat Enterprise Linux" 2022-02-11T17:21:11.424 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="8.4 (Ootpa)" 2022-02-11T17:21:11.424 INFO:teuthology.orchestra.run.smithi124.stdout:ID="rhel" 2022-02-11T17:21:11.424 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE="fedora" 2022-02-11T17:21:11.424 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="8.4" 2022-02-11T17:21:11.425 INFO:teuthology.orchestra.run.smithi124.stdout:PLATFORM_ID="platform:el8" 2022-02-11T17:21:11.426 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-02-11T17:21:11.426 INFO:teuthology.orchestra.run.smithi124.stdout:ANSI_COLOR="0;31" 2022-02-11T17:21:11.426 INFO:teuthology.orchestra.run.smithi124.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-02-11T17:21:11.426 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://www.redhat.com/" 2022-02-11T17:21:11.427 INFO:teuthology.orchestra.run.smithi124.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-02-11T17:21:11.427 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-11T17:21:11.427 INFO:teuthology.orchestra.run.smithi124.stdout: 2022-02-11T17:21:11.427 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-11T17:21:11.428 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-02-11T17:21:11.428 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-02-11T17:21:11.428 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-02-11T17:21:11.430 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2022-02-11T17:21:11.449 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: set os type and version on server 2022-02-11T17:21:11.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2022-02-11T17:21:11.676 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2022-02-11T17:21:11.730 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="Red Hat Enterprise Linux" 2022-02-11T17:21:11.730 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="8.4 (Ootpa)" 2022-02-11T17:21:11.731 INFO:teuthology.orchestra.run.smithi183.stdout:ID="rhel" 2022-02-11T17:21:11.731 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="fedora" 2022-02-11T17:21:11.731 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="8.4" 2022-02-11T17:21:11.731 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el8" 2022-02-11T17:21:11.731 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-02-11T17:21:11.732 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2022-02-11T17:21:11.732 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-02-11T17:21:11.732 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://www.redhat.com/" 2022-02-11T17:21:11.732 INFO:teuthology.orchestra.run.smithi183.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-02-11T17:21:11.733 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-11T17:21:11.733 INFO:teuthology.orchestra.run.smithi183.stdout: 2022-02-11T17:21:11.733 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-11T17:21:11.733 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-02-11T17:21:11.734 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-02-11T17:21:11.734 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-02-11T17:21:11.734 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2022-02-11T17:21:11.833 INFO:teuthology.dispatcher.supervisor:Running job 6676865 2022-02-11T17:21:11.834 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2022-02-11_16:35:44-orch:cephadm-wip-adk-testing-2022-02-10-1637-distro-basic-smithi/6676865 --name adking-2022-02-11_16:35:44-orch:cephadm-wip-adk-testing-2022-02-10-1637-distro-basic-smithi --description orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} -- /home/teuthworker/archive/adking-2022-02-11_16:35:44-orch:cephadm-wip-adk-testing-2022-02-10-1637-distro-basic-smithi/6676865/orig.config.yaml 2022-02-11T17:21:11.837 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2022-02-11_16:35:44-orch:cephadm-wip-adk-testing-2022-02-10-1637-distro-basic-smithi/6676865 2022-02-11T17:21:11.838 INFO:teuthology.dispatcher.supervisor:Job PID: 4848 2022-02-11T17:21:11.838 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-11T23:51:40.367 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-11T23:51:40.591 INFO:teuthology.kill:Killing Pids: {4848} 2022-02-11T23:51:41.784 INFO:teuthology.task.internal:roles: ubuntu@smithi124.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-02-11T23:51:41.785 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-02-11T23:51:41.785 INFO:teuthology.misc:Compressing logs... 2022-02-11T23:51:41.786 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2022-02-11T23:51:41.979 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2022-02-11T23:51:42.025 INFO:teuthology.orchestra.run.smithi124.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-02-11T23:51:42.254 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-02-11T23:51:42.287 INFO:teuthology.orchestra.run.smithi124.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-02-11T23:51:42.287 INFO:teuthology.orchestra.run.smithi124.stderr:tar: Error is not recoverable: exiting now 2022-02-11T23:51:42.313 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-02-11T23:51:42.314 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2022-02-11T23:51:42.315 INFO:teuthology.misc:Compressing logs... 2022-02-11T23:51:42.510 INFO:teuthology.misc:Compressing logs... 2022-02-11T23:51:49.770 INFO:teuthology.orchestra.run.smithi124.stderr:gzip: /var/log/ceph/68d380d2-8b60-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-02-11T23:51:51.464 INFO:teuthology.orchestra.run.smithi124.stderr:gzip: /var/log/ceph/68d380d2-8b60-11ec-8c35-001a4aab830c/ceph-mgr.smithi124.edhvex.log: file size changed while zipping 2022-02-11T23:51:51.982 INFO:teuthology.orchestra.run.smithi183.stderr:gzip: /var/log/ceph/68d380d2-8b60-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-02-11T23:51:58.720 INFO:teuthology.orchestra.run.smithi124.stderr:gzip: /var/log/ceph/68d380d2-8b60-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-02-11T23:51:58.830 INFO:teuthology.orchestra.run.smithi183.stderr:gzip: /var/log/ceph/68d380d2-8b60-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-02-11T23:52:06.749 INFO:teuthology.orchestra.run.smithi124.stderr:gzip: /var/log/ceph/68d380d2-8b60-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-02-11T23:52:08.189 INFO:teuthology.orchestra.run.smithi183.stderr:gzip: /var/log/ceph/68d380d2-8b60-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-02-11T23:52:11.715 INFO:teuthology.orchestra.run.smithi183.stderr:gzip: /var/log/ceph/68d380d2-8b60-11ec-8c35-001a4aab830c/ceph-mon.smithi183.log: file size changed while zipping 2022-02-11T23:52:16.459 INFO:teuthology.orchestra.run.smithi124.stderr:gzip: /var/log/ceph/68d380d2-8b60-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-02-11T23:52:18.935 INFO:teuthology.orchestra.run.smithi183.stderr:gzip: /var/log/ceph/68d380d2-8b60-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-02-11T23:52:25.660 INFO:teuthology.orchestra.run.smithi124.stderr:gzip: /var/log/ceph/68d380d2-8b60-11ec-8c35-001a4aab830c/ceph-mon.smithi124.log: file size changed while zipping 2022-02-11T23:52:42.783 INFO:teuthology.kill:No teuthology processes running 2022-02-11T23:52:44.123 INFO:teuthology.kill:Nuking machines: ['smithi124', 'smithi183'] 2022-02-11T23:52:45.372 INFO:teuthology.kill:2022-02-11 23:52:45,372.372 INFO:teuthology.nuke:targets: 2022-02-11T23:52:45.373 INFO:teuthology.kill: smithi124.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDsz4wl+fM2gJoF3u65VDzzpFGKJwUYZUvoqSJhwjNhcugzVmjrr7WGAFQyyV1JlNfi0kPZSRuYjfsETDYvZZw53yL+PYuLGWNda+Az/JWmGkB5GmMAhvP72tZJNENv0oCVQkxuBLLrRgAqfWOa5tCD7tjGK7G72Vly2jycBIRceBSAKp+Z1Dc2IvQk25PiKVON4iVmhJT+kjl7CEOCbVOBXok9J3bh6wE0Ayw2ShNhfuA3XA41IXztXJYrp8MUL35fZnSXlLv9dbwLdK3Rf/F73XzMb3o3YwlU151secPoLjJ0NGMaaVChddmBed+H4spZ7vchgc8zPOZzziX7fsLJTkuf1ibpd5huQNnddIkgfojM58Ky+9sR7yp97A9B+dOzsteBbys56QZ0VzspPiQeUt00JUZDpWsLiswSugx6cyObGrwYAqedNMPFgNfGKv5Bny+mwmDskC4HSrykLYcHBQ+YAI+/OlJUOUAj1hB6zGJtfFM2mmMBJQhGVyKtqf0= 2022-02-11T23:52:45.373 INFO:teuthology.kill: smithi183.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDe6FfHX3cTEvmQAPOb2tji75umF+t3fsgx3VAiDMPmR2Jlk+wBNSpCWXtxybutyC4SmQCvIIoy4iAZN2TaUcrs8Wr17WylKADXifII3MdSX/M8uNIhJ+cQyqUjfxrLy6UZFwyGKfrsj2/9Jo4yLsABW7bhkCdw5dHF779iCaxQN4VgaSZJney9QnuCCzjaOSAMFT/oDF6cpjlBXsBIB4pdW6V3IRcK5cBcoLHY6O/ATADlSzyyWwbvsPeDYu2PMrtGm88kPeJArwyHz5ayqnrCFx5cBK2081lVcgB9J0/xfOQTu+fIXpNahthuYrcUibtmmLnI7xtgYvGSha3QN/1bgDb0IbZxcBDuiNJFmCeI/pIziULRMXPCheEt4GZG1WqUqVRKX63aVvDTdEhLUNtffFKBLnkunVyoGoaw4SlLEG6+ZxGw/nlmJwIEgW4fm60T0HYuwxg27ItU3SFaRK3HpMRDGQ0TkFnuKOy1GlRNp6yNZaGlLGTEV7zvfkvH+Sk= 2022-02-11T23:52:45.400 INFO:teuthology.kill:2022-02-11 23:52:45,399.399 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-11T23:52:45.403 INFO:teuthology.kill:2022-02-11 23:52:45,402.402 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-11T23:52:45.478 INFO:teuthology.kill:2022-02-11 23:52:45,478.478 INFO:teuthology.orchestra.console:Power off smithi183 2022-02-11T23:52:45.491 INFO:teuthology.kill:2022-02-11 23:52:45,491.491 INFO:teuthology.orchestra.console:Power off smithi124 2022-02-11T23:52:53.857 INFO:teuthology.kill:2022-02-11 23:52:53,857.857 INFO:teuthology.orchestra.console:Power off for smithi124 completed 2022-02-11T23:52:54.002 INFO:teuthology.kill:2022-02-11 23:52:54,002.002 INFO:teuthology.lock.ops:unlocked: smithi124.front.sepia.ceph.com 2022-02-11T23:52:57.962 INFO:teuthology.kill:2022-02-11 23:52:57,961.961 INFO:teuthology.orchestra.console:Power off for smithi183 completed 2022-02-11T23:52:58.108 INFO:teuthology.kill:2022-02-11 23:52:58,108.108 INFO:teuthology.lock.ops:unlocked: smithi183.front.sepia.ceph.com 2022-02-11T23:54:58.465 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-11T23:54:58.571 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi124 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-11_19:41:29-rados-wip-yuriw-quincy-RC-2.11.22-distro-default-smithi/6677395 2022-02-11T23:54:58.571 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi183 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-11_19:41:29-rados-wip-yuriw-quincy-RC-2.11.22-distro-default-smithi/6677393