2022-01-24T22:03:05.887 INFO:root:teuthology version: 1.1.0-7c0cb867 2022-01-24T22:03:06.002 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi017.front.sepia.ceph.com' reimaging 2022-01-24T22:03:06.003 INFO:teuthology.lock.ops:Updating [ubuntu@smithi017.front.sepia.ceph.com]: reset os type and version on server 2022-01-24T22:03:06.003 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2022-01-24T22:03:06.021 INFO:teuthology.lock.ops:Node 'ubuntu@smithi017.front.sepia.ceph.com' reimaging is complete 2022-01-24T22:03:06.022 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi089.front.sepia.ceph.com' reimaging 2022-01-24T22:03:06.022 INFO:teuthology.lock.ops:Updating [ubuntu@smithi089.front.sepia.ceph.com]: reset os type and version on server 2022-01-24T22:03:06.022 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2022-01-24T22:03:06.041 INFO:teuthology.lock.ops:Node 'ubuntu@smithi089.front.sepia.ceph.com' reimaging is complete 2022-01-24T22:03:06.236 INFO:teuthology.provision.fog.smithi089:Scheduling deploy of centos 8.2 2022-01-24T22:03:06.240 INFO:teuthology.provision.fog.smithi017:Scheduling deploy of centos 8.2 2022-01-24T22:03:06.581 INFO:teuthology.orchestra.console:Power off smithi017 2022-01-24T22:03:06.613 INFO:teuthology.orchestra.console:Power off smithi089 2022-01-24T22:03:14.951 INFO:teuthology.orchestra.console:Power off for smithi017 completed 2022-01-24T22:03:15.055 INFO:teuthology.orchestra.console:Power on smithi017 2022-01-24T22:03:19.087 INFO:teuthology.orchestra.console:Power off for smithi089 completed 2022-01-24T22:03:19.188 INFO:teuthology.orchestra.console:Power on smithi089 2022-01-24T22:03:19.317 INFO:teuthology.orchestra.console:Power on for smithi017 completed 2022-01-24T22:03:19.418 INFO:teuthology.provision.fog.smithi017:Waiting for deploy to finish 2022-01-24T22:03:23.438 INFO:teuthology.orchestra.console:Power on for smithi089 completed 2022-01-24T22:03:23.539 INFO:teuthology.provision.fog.smithi089:Waiting for deploy to finish 2022-01-24T22:05:35.665 ERROR:teuthology.orchestra.connection:Error authenticating with smithi017.front.sepia.ceph.com: Authentication failed. 2022-01-24T22:07:09.615 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-24T22:07:25.709 INFO:teuthology.provision.fog.smithi017:Node is ready 2022-01-24T22:07:25.728 INFO:teuthology.orchestra.run.smithi017.stdout:smithi017.front.sepia.ceph.com 2022-01-24T22:07:25.785 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17 smithi017.front.sepia.ceph.com smithi017 2022-01-24T22:07:26.140 INFO:teuthology.provision.fog.smithi017:Deploy complete! 2022-01-24T22:07:36.123 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-24T22:07:56.447 INFO:teuthology.provision.fog.smithi089:Node is ready 2022-01-24T22:07:56.466 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089.front.sepia.ceph.com 2022-01-24T22:07:56.522 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89 smithi089.front.sepia.ceph.com smithi089 2022-01-24T22:07:56.886 INFO:teuthology.provision.fog.smithi089:Deploy complete! 2022-01-24T22:07:57.066 INFO:teuthology.lock.ops:Checking smithi017.front.sepia.ceph.com 2022-01-24T22:07:57.067 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-24T22:07:57.086 INFO:teuthology.lock.ops:Checking smithi089.front.sepia.ceph.com 2022-01-24T22:07:57.087 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-24T22:07:57.104 INFO:teuthology.lock.ops:Updating [smithi017.front.sepia.ceph.com]: set os type and version on server 2022-01-24T22:07:57.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2022-01-24T22:07:57.377 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64 2022-01-24T22:07:57.432 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="CentOS Linux" 2022-01-24T22:07:57.433 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="8 (Core)" 2022-01-24T22:07:57.433 INFO:teuthology.orchestra.run.smithi017.stdout:ID="centos" 2022-01-24T22:07:57.433 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="rhel fedora" 2022-01-24T22:07:57.433 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="8" 2022-01-24T22:07:57.434 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el8" 2022-01-24T22:07:57.434 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2022-01-24T22:07:57.434 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31" 2022-01-24T22:07:57.434 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-24T22:07:57.434 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://www.centos.org/" 2022-01-24T22:07:57.435 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-24T22:07:57.435 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-01-24T22:07:57.435 INFO:teuthology.orchestra.run.smithi017.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-24T22:07:57.435 INFO:teuthology.orchestra.run.smithi017.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-24T22:07:57.435 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2022-01-24T22:07:57.436 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2022-01-24T22:07:57.436 INFO:teuthology.orchestra.run.smithi017.stdout: 2022-01-24T22:07:57.437 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2022-01-24T22:07:57.465 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: set os type and version on server 2022-01-24T22:07:57.466 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2022-01-24T22:07:57.730 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2022-01-24T22:07:57.788 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Linux" 2022-01-24T22:07:57.788 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8 (Core)" 2022-01-24T22:07:57.788 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2022-01-24T22:07:57.789 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2022-01-24T22:07:57.789 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8" 2022-01-24T22:07:57.789 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8" 2022-01-24T22:07:57.789 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2022-01-24T22:07:57.790 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2022-01-24T22:07:57.790 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-24T22:07:57.790 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://www.centos.org/" 2022-01-24T22:07:57.790 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2022-01-24T22:07:57.791 INFO:teuthology.orchestra.run.smithi089.stdout: 2022-01-24T22:07:57.791 INFO:teuthology.orchestra.run.smithi089.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2022-01-24T22:07:57.791 INFO:teuthology.orchestra.run.smithi089.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2022-01-24T22:07:57.791 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2022-01-24T22:07:57.791 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2022-01-24T22:07:57.792 INFO:teuthology.orchestra.run.smithi089.stdout: 2022-01-24T22:07:57.792 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2022-01-24T22:07:57.879 INFO:teuthology.dispatcher.supervisor:Running job 6637924 2022-01-24T22:07:57.879 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7c0cb8672986d9dbe53078a123af65593653ef7a/virtualenv/bin/teuthology -v --owner scheduled_swagner@teuthology --archive /home/teuthworker/archive/swagner-2022-01-24_16:47:41-orch:cephadm-wip-swagner-testing-2022-01-24-1121-distro-default-smithi/6637924 --name swagner-2022-01-24_16:47:41-orch:cephadm-wip-swagner-testing-2022-01-24-1121-distro-default-smithi --description orch:cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} -- /home/teuthworker/archive/swagner-2022-01-24_16:47:41-orch:cephadm-wip-swagner-testing-2022-01-24-1121-distro-default-smithi/6637924/orig.config.yaml 2022-01-24T22:07:57.882 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/swagner-2022-01-24_16:47:41-orch:cephadm-wip-swagner-testing-2022-01-24-1121-distro-default-smithi/6637924 2022-01-24T22:07:57.883 INFO:teuthology.dispatcher.supervisor:Job PID: 6500 2022-01-24T22:07:57.884 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-25T04:38:08.797 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-25T04:38:08.970 INFO:teuthology.kill:Killing Pids: {6500} 2022-01-25T04:38:10.159 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-25T04:38:10.160 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-25T04:38:10.160 INFO:teuthology.misc:Compressing logs... 2022-01-25T04:38:10.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2022-01-25T04:38:10.369 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2022-01-25T04:38:10.418 INFO:teuthology.orchestra.run.smithi017.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-25T04:38:10.725 INFO:teuthology.orchestra.run.smithi089.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-25T04:38:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-25T04:38:10.760 INFO:teuthology.orchestra.run.smithi017.stderr:tar: Error is not recoverable: exiting now 2022-01-25T04:38:10.790 INFO:teuthology.orchestra.run.smithi089.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-25T04:38:10.791 INFO:teuthology.orchestra.run.smithi089.stderr:tar: Error is not recoverable: exiting now 2022-01-25T04:38:10.792 INFO:teuthology.misc:Compressing logs... 2022-01-25T04:38:10.996 INFO:teuthology.misc:Compressing logs... 2022-01-25T04:38:12.121 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/5ab44d66-7d63-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-25T04:38:12.268 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/5ab44d66-7d63-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-25T04:38:12.368 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/5ab44d66-7d63-11ec-8c35-001a4aab830c/ceph-mgr.smithi017.phpxbm.log: file size changed while zipping 2022-01-25T04:38:13.196 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/5ab44d66-7d63-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-25T04:38:13.459 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/5ab44d66-7d63-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-25T04:38:14.179 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/5ab44d66-7d63-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-25T04:38:14.816 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/5ab44d66-7d63-11ec-8c35-001a4aab830c/ceph-mon.smithi017.log: file size changed while zipping 2022-01-25T04:38:15.355 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/5ab44d66-7d63-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-25T04:38:15.823 INFO:teuthology.orchestra.run.smithi089.stderr:gzip: /var/log/ceph/5ab44d66-7d63-11ec-8c35-001a4aab830c/ceph-mon.smithi089.log: file size changed while zipping 2022-01-25T04:38:16.023 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/5ab44d66-7d63-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-25T04:38:17.483 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/5ab44d66-7d63-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-25T04:38:34.969 INFO:teuthology.kill:No teuthology processes running 2022-01-25T04:38:36.255 INFO:teuthology.kill:Nuking machines: ['smithi017', 'smithi089'] 2022-01-25T04:38:37.246 INFO:teuthology.kill:2022-01-25 04:38:37,245.245 INFO:teuthology.nuke:targets: 2022-01-25T04:38:37.247 INFO:teuthology.kill: smithi017.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGzJSUFYJ0YNP3aPepErH6TCai9NuakM6thUiSK7Ij5P23iVTAnNmn/ucwQfCAlba+D7b2SwhHks304W30Se7ppavk0shjU7CXi1YmxYzRutJ1lWOeqsPQQz5L2xAdPCNWdo9KlMlkslLISFthU4cisd8cm+4KIbi5SSbknNef3zSmi7sQ6Egvl+0EX9jBbUv9jtRTO2P3rlrQ4fMwGUsONLB03rMTQkehFgg1cAuT5Uk/rLdQo3+r/iGdspi9IxjNE6DiUkDPSNzXd1rNS+3AZE5RM/QiRfdzlc6yU4O98V2KOpt3v3cZhvCq6WQTIzUkAyJlqHm4QbatcIB5pX31M3XkZwM0qCPpP0FMYOAYFkH76mcDLpJF5rSRystHF/caTX8+sBL3+tN7AgC3ANiLkglWmw68vOqX7QAXCdrgqTbPpjwqVliquP02gjlw+mzKbal9mbxmLXyP4ka/VXKWHzdl3CllQZ5GGsSnt5KMpnTJQUDQDtIBSRY9Uw5UtGs= 2022-01-25T04:38:37.247 INFO:teuthology.kill: smithi089.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdwQzDaUmMsKVSlNSXd2/ibXl0ybwvFPudEN4ujrGLu6qW47189+VdYEg2AtIw72VXEo6yTsIAdnHIPx6zWlGhQSt/HeXmOyY3/mqZX/zKF8Oh7qOZz+cR6VdruAQMyFK/IAOYPIbfI1xXfz2mdn7vogDqtyuROt2JSQSdoOKu2fcdzwsoEl+HruQn9MEhoBhdPUUDnwoZxiSvVbM6L8ft0dDKLtPCfJRKSGfd9Yb2TTpXmF+lzN+IHjTCXXFh3JezuhgQx/YO/WFZWaygIoN85X7KG4SjY3ztVdmteVOhr9BYqPNiLYfHJcksGNayMvLGc/8obnqeKkNRIuhBTAU1kvVGqK9XIIsfBKjAU05222TG7SvrFHafWSSajnU4zBImqZBPPmmc2Aawb3wKPLUrfA+Kn/cF1MAzTJAAw6rHgOh0crh7KUVhtGCP9vCqQ4S/URZgq12Hvmm8Bng5SRNwBz6BnpuGJsJCQfAjAgQZ7zBE38TqGMOpjSar0vRjPvk= 2022-01-25T04:38:37.273 INFO:teuthology.kill:2022-01-25 04:38:37,273.273 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-25T04:38:37.278 INFO:teuthology.kill:2022-01-25 04:38:37,278.278 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-25T04:38:37.357 INFO:teuthology.kill:2022-01-25 04:38:37,356.356 INFO:teuthology.orchestra.console:Power off smithi089 2022-01-25T04:38:37.374 INFO:teuthology.kill:2022-01-25 04:38:37,373.373 INFO:teuthology.orchestra.console:Power off smithi017 2022-01-25T04:38:45.735 INFO:teuthology.kill:2022-01-25 04:38:45,734.734 INFO:teuthology.orchestra.console:Power off for smithi089 completed 2022-01-25T04:38:45.880 INFO:teuthology.kill:2022-01-25 04:38:45,879.879 INFO:teuthology.lock.ops:unlocked: smithi089.front.sepia.ceph.com 2022-01-25T04:38:49.859 INFO:teuthology.kill:2022-01-25 04:38:49,858.858 INFO:teuthology.orchestra.console:Power off for smithi017 completed 2022-01-25T04:38:50.004 INFO:teuthology.kill:2022-01-25 04:38:50,004.004 INFO:teuthology.lock.ops:unlocked: smithi017.front.sepia.ceph.com 2022-01-25T04:40:50.227 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-25T04:40:50.320 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi017 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-25_03:40:28-rados-wip-yuri4-testing-2022-01-24-1706-pacific-distro-default-smithi/6639114 2022-01-25T04:40:50.321 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi089 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-25_03:40:28-rados-wip-yuri4-testing-2022-01-24-1706-pacific-distro-default-smithi/6639114