2022-02-11T18:29:45.629 INFO:root:teuthology version: 1.1.0-7cebb3f2 2022-02-11T18:29:45.749 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi162.front.sepia.ceph.com' reimaging 2022-02-11T18:29:45.750 INFO:teuthology.lock.ops:Updating [ubuntu@smithi162.front.sepia.ceph.com]: reset os type and version on server 2022-02-11T18:29:45.750 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2022-02-11T18:29:45.772 INFO:teuthology.lock.ops:Node 'ubuntu@smithi162.front.sepia.ceph.com' reimaging is complete 2022-02-11T18:29:45.772 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi179.front.sepia.ceph.com' reimaging 2022-02-11T18:29:45.772 INFO:teuthology.lock.ops:Updating [ubuntu@smithi179.front.sepia.ceph.com]: reset os type and version on server 2022-02-11T18:29:45.773 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2022-02-11T18:29:45.793 INFO:teuthology.lock.ops:Node 'ubuntu@smithi179.front.sepia.ceph.com' reimaging is complete 2022-02-11T18:29:46.001 INFO:teuthology.provision.fog.smithi162:Scheduling deploy of centos 8.stream 2022-02-11T18:29:46.005 INFO:teuthology.provision.fog.smithi179:Scheduling deploy of centos 8.stream 2022-02-11T18:29:46.332 INFO:teuthology.orchestra.console:Power off smithi179 2022-02-11T18:29:46.464 INFO:teuthology.orchestra.console:Power off smithi162 2022-02-11T18:29:54.701 INFO:teuthology.orchestra.console:Power off for smithi179 completed 2022-02-11T18:29:54.803 INFO:teuthology.orchestra.console:Power on smithi179 2022-02-11T18:29:54.823 INFO:teuthology.orchestra.console:Power off for smithi162 completed 2022-02-11T18:29:54.924 INFO:teuthology.orchestra.console:Power on smithi162 2022-02-11T18:30:03.182 INFO:teuthology.orchestra.console:Power on for smithi179 completed 2022-02-11T18:30:03.283 INFO:teuthology.orchestra.console:Power on for smithi162 completed 2022-02-11T18:30:03.283 INFO:teuthology.provision.fog.smithi179:Waiting for deploy to finish 2022-02-11T18:30:03.384 INFO:teuthology.provision.fog.smithi162:Waiting for deploy to finish 2022-02-11T18:34:24.630 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-11T18:34:34.997 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-11T18:34:35.231 INFO:teuthology.provision.fog.smithi179:Node is ready 2022-02-11T18:34:35.248 INFO:teuthology.orchestra.run.smithi179.stdout:smithi179.front.sepia.ceph.com 2022-02-11T18:34:35.307 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179 smithi179.front.sepia.ceph.com smithi179 2022-02-11T18:34:35.645 INFO:teuthology.provision.fog.smithi179:Deploy complete! 2022-02-11T18:34:49.781 INFO:teuthology.provision.fog.smithi162:Node is ready 2022-02-11T18:34:49.841 INFO:teuthology.orchestra.run.smithi162.stdout:smithi162.front.sepia.ceph.com 2022-02-11T18:34:49.901 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162 smithi162.front.sepia.ceph.com smithi162 2022-02-11T18:34:50.287 INFO:teuthology.provision.fog.smithi162:Deploy complete! 2022-02-11T18:34:50.445 INFO:teuthology.lock.ops:Checking smithi162.front.sepia.ceph.com 2022-02-11T18:34:50.446 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-11T18:34:50.466 INFO:teuthology.lock.ops:Checking smithi179.front.sepia.ceph.com 2022-02-11T18:34:50.466 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-11T18:34:50.485 INFO:teuthology.lock.ops:Updating [smithi162.front.sepia.ceph.com]: set os type and version on server 2022-02-11T18:34:50.486 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2022-02-11T18:34:50.678 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2022-02-11T18:34:50.735 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream" 2022-02-11T18:34:50.735 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="8" 2022-02-11T18:34:50.735 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos" 2022-02-11T18:34:50.735 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora" 2022-02-11T18:34:50.736 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="8" 2022-02-11T18:34:50.736 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el8" 2022-02-11T18:34:50.736 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 8" 2022-02-11T18:34:50.736 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2022-02-11T18:34:50.737 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-02-11T18:34:50.737 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/" 2022-02-11T18:34:50.737 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-11T18:34:50.737 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-11T18:34:50.737 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-02-11T18:34:50.739 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2022-02-11T18:34:50.757 INFO:teuthology.lock.ops:Updating [smithi179.front.sepia.ceph.com]: set os type and version on server 2022-02-11T18:34:50.758 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2022-02-11T18:34:51.006 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2022-02-11T18:34:51.064 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream" 2022-02-11T18:34:51.064 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="8" 2022-02-11T18:34:51.064 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos" 2022-02-11T18:34:51.065 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora" 2022-02-11T18:34:51.065 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="8" 2022-02-11T18:34:51.065 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el8" 2022-02-11T18:34:51.065 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 8" 2022-02-11T18:34:51.066 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2022-02-11T18:34:51.066 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-02-11T18:34:51.066 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/" 2022-02-11T18:34:51.066 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-11T18:34:51.067 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-11T18:34:51.067 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-02-11T18:34:51.067 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2022-02-11T18:34:51.169 INFO:teuthology.dispatcher.supervisor:Running job 6676925 2022-02-11T18:34:51.169 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2022-02-11_16:35:44-orch:cephadm-wip-adk-testing-2022-02-10-1637-distro-basic-smithi/6676925 --name adking-2022-02-11_16:35:44-orch:cephadm-wip-adk-testing-2022-02-10-1637-distro-basic-smithi --description orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} -- /home/teuthworker/archive/adking-2022-02-11_16:35:44-orch:cephadm-wip-adk-testing-2022-02-10-1637-distro-basic-smithi/6676925/orig.config.yaml 2022-02-11T18:34:51.173 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2022-02-11_16:35:44-orch:cephadm-wip-adk-testing-2022-02-10-1637-distro-basic-smithi/6676925 2022-02-11T18:34:51.174 INFO:teuthology.dispatcher.supervisor:Job PID: 26208 2022-02-11T18:34:51.174 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-12T01:05:19.876 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-12T01:05:20.128 INFO:teuthology.kill:Killing Pids: {26208} 2022-02-12T01:05:21.291 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-02-12T01:05:21.291 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-02-12T01:05:21.292 INFO:teuthology.misc:Compressing logs... 2022-02-12T01:05:21.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2022-02-12T01:05:21.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2022-02-12T01:05:21.538 INFO:teuthology.orchestra.run.smithi162.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-02-12T01:05:21.730 INFO:teuthology.orchestra.run.smithi179.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-02-12T01:05:21.764 INFO:teuthology.orchestra.run.smithi162.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-02-12T01:05:21.764 INFO:teuthology.orchestra.run.smithi162.stderr:tar: Error is not recoverable: exiting now 2022-02-12T01:05:21.796 INFO:teuthology.orchestra.run.smithi179.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-02-12T01:05:21.796 INFO:teuthology.orchestra.run.smithi179.stderr:tar: Error is not recoverable: exiting now 2022-02-12T01:05:21.798 INFO:teuthology.misc:Compressing logs... 2022-02-12T01:05:22.000 INFO:teuthology.misc:Compressing logs... 2022-02-12T01:05:29.467 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/149336f2-8b6a-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-02-12T01:05:31.537 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/149336f2-8b6a-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-02-12T01:05:35.027 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/149336f2-8b6a-11ec-8c35-001a4aab830c/ceph-mon.smithi179.log: file size changed while zipping 2022-02-12T01:05:37.644 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/149336f2-8b6a-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-02-12T01:05:39.437 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/149336f2-8b6a-11ec-8c35-001a4aab830c/ceph-mgr.smithi162.nqmwjm.log: file size changed while zipping 2022-02-12T01:05:42.095 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/149336f2-8b6a-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-02-12T01:05:49.284 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/149336f2-8b6a-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-02-12T01:05:51.647 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/149336f2-8b6a-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-02-12T01:05:58.712 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/149336f2-8b6a-11ec-8c35-001a4aab830c/ceph-mon.smithi162.log: file size changed while zipping 2022-02-12T01:05:59.103 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/149336f2-8b6a-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-02-12T01:06:06.127 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/149336f2-8b6a-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-02-12T01:06:23.723 INFO:teuthology.kill:No teuthology processes running 2022-02-12T01:06:24.879 INFO:teuthology.kill:Nuking machines: ['smithi162', 'smithi179'] 2022-02-12T01:06:25.885 INFO:teuthology.kill:2022-02-12 01:06:25,885.885 INFO:teuthology.nuke:targets: 2022-02-12T01:06:25.887 INFO:teuthology.kill: smithi162.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsMt4+ECUKvlj7pPNlqADB0JacE+tg85DsZW51aJjtG0Ceuz6CPSLOCc5kSLqsI3NQREYeDqB125bABAkLxE0XWpK6T2Ips00OkcbdvSo/c9pmJpDRxLpBq/Xtu0d/RMvGtG7qchsu+4uUJVXt9fF6tflF+ZCGk9LRbzdOU19lGJyPG493ObkCTyRkr81eMYnZf6sv7DGv556xWBsx083WsUhs5tiiRtMeJoUgjA7EjbcRb9sqqTgFS5TLoxO16dVt+nAXeggDZBvayH/ahP0tzHV5OJzxqjOIDUr9xGSoqnS0nBhexrMpvKb2vUHSob26XBQdRBWGk+mrpL8XQ8lNMXbGNK2aHpvbX1GgZMFwdfD8dc1xgwXUCvH+XbhFK1VnHnaLlmUespPuEj2N20yyplSUs681GDBnla3NQNQ3oeMHrQ5uQBdApFCQtTDabm/iEO/MBJVnw9IForYLv55q6w6ZC2c82tqAcQbBMO4hoFaVaJMvuj01W5txo3Bugic= 2022-02-12T01:06:25.887 INFO:teuthology.kill: smithi179.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChAqwGA2NEQua/5EgBHv+WsmwuLYAZgjHv57lCKiZDEBU7a9/ikU/f2JgSpI7XGq/nn7xAoTPNYI/xicmj5LqQoGZPIzudHFxZVfD8Gj+HpvQ7yprCvRPEcSFGyIUsnaWnk9giYEQyCPX5mJs2qVeKKVAp3bdWxWSHEsPZ50O5/Rl7nUMLU+cGAh27dSMPIhlPa4KVrFgDkv/r0kQj/0uiZTLP8rYSp1NTwV6VPhYH3e+BBxWR6GqjEwNZhHP9CtQVzOSO1JGLjtdCkrhMJp6WOTHDgBnIQVK5q9WCkGn98j3Roew+khLneLWkaiaLyDHBmQiVNEZd9X2p9meks58Sy3YFj/UtTU1ivYN/LsxjkRe2oBXI2j0Rs0mlUAbjglHdezpiGz8G+rjwgrE6xzHvNMrKc+ACsf1MqlESD0cDIELx5s5XYemen3mItmiUFauI4AfKv0MmmvC8HP4Chk07n3T7KEhw9cxdjfUkyVXPRiPyjSaOcB1ePHIUL6NSvMk= 2022-02-12T01:06:25.914 INFO:teuthology.kill:2022-02-12 01:06:25,913.913 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-12T01:06:25.918 INFO:teuthology.kill:2022-02-12 01:06:25,917.917 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-12T01:06:26.002 INFO:teuthology.kill:2022-02-12 01:06:26,001.001 INFO:teuthology.orchestra.console:Power off smithi179 2022-02-12T01:06:26.015 INFO:teuthology.kill:2022-02-12 01:06:26,014.014 INFO:teuthology.orchestra.console:Power off smithi162 2022-02-12T01:06:34.372 INFO:teuthology.kill:2022-02-12 01:06:34,372.372 INFO:teuthology.orchestra.console:Power off for smithi179 completed 2022-02-12T01:06:34.385 INFO:teuthology.kill:2022-02-12 01:06:34,385.385 INFO:teuthology.orchestra.console:Power off for smithi162 completed 2022-02-12T01:06:34.523 INFO:teuthology.kill:2022-02-12 01:06:34,522.522 INFO:teuthology.lock.ops:unlocked: smithi179.front.sepia.ceph.com 2022-02-12T01:06:34.529 INFO:teuthology.kill:2022-02-12 01:06:34,529.529 INFO:teuthology.lock.ops:unlocked: smithi162.front.sepia.ceph.com 2022-02-12T01:08:34.826 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-12T01:08:34.924 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi162 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-11_19:41:29-rados-wip-yuriw-quincy-RC-2.11.22-distro-default-smithi/6677520 2022-02-12T01:08:34.925 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi179 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-11_19:41:29-rados-wip-yuriw-quincy-RC-2.11.22-distro-default-smithi/6677521