2023-01-12T12:47:11.410 INFO:root:teuthology version: 0.0.1.dev18+g19d18db 2023-01-12T12:47:23.732 INFO:teuthology.lock.ops:Start node 'smithi016.front.sepia.ceph.com' reimaging 2023-01-12T12:47:23.735 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: reset os type and version on server 2023-01-12T12:47:23.736 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-01-12T12:47:23.752 INFO:teuthology.lock.ops:Node 'smithi016.front.sepia.ceph.com' reimaging is complete 2023-01-12T12:47:23.752 INFO:teuthology.lock.ops:Start node 'smithi064.front.sepia.ceph.com' reimaging 2023-01-12T12:47:23.753 INFO:teuthology.lock.ops:Updating [smithi064.front.sepia.ceph.com]: reset os type and version on server 2023-01-12T12:47:23.753 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2023-01-12T12:47:28.777 INFO:teuthology.lock.ops:Node 'smithi064.front.sepia.ceph.com' reimaging is complete 2023-01-12T12:47:48.227 INFO:teuthology.provision.fog.smithi016:Scheduling deploy of centos 8.stream 2023-01-12T12:47:48.311 INFO:teuthology.provision.fog.smithi064:Scheduling deploy of centos 8.stream 2023-01-12T12:47:48.488 INFO:teuthology.orchestra.console:Power off smithi016 2023-01-12T12:47:48.530 INFO:teuthology.orchestra.console:Power off smithi064 2023-01-12T12:48:02.986 INFO:teuthology.orchestra.console:Power off for smithi016 completed 2023-01-12T12:48:03.101 INFO:teuthology.orchestra.console:Power on smithi016 2023-01-12T12:48:03.649 INFO:teuthology.orchestra.console:Power off for smithi064 completed 2023-01-12T12:48:03.750 INFO:teuthology.orchestra.console:Power on smithi064 2023-01-12T12:48:12.864 INFO:teuthology.orchestra.console:Power on for smithi016 completed 2023-01-12T12:48:12.974 INFO:teuthology.provision.fog.smithi016:Waiting for deploy to finish 2023-01-12T12:48:13.124 INFO:teuthology.orchestra.console:Power on for smithi064 completed 2023-01-12T12:48:13.225 INFO:teuthology.provision.fog.smithi064:Waiting for deploy to finish 2023-01-12T12:50:54.541 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-01-12T12:51:03.628 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-01-12T12:51:09.648 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-01-12T12:51:12.688 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-01-12T12:51:18.732 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-01-12T12:51:21.777 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-01-12T12:51:27.792 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-01-12T12:51:36.876 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-01-12T12:51:45.968 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.64 2023-01-12T12:51:59.943 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-01-12T12:52:14.875 INFO:teuthology.provision.fog.smithi016:Node is ready 2023-01-12T12:52:14.898 INFO:teuthology.orchestra.run.smithi016.stdout:smithi001 2023-01-12T12:52:14.963 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.1 smithi001 smithi001 2023-01-12T12:52:15.345 INFO:teuthology.provision.fog.smithi016:Deploy complete! 2023-01-12T12:52:52.012 WARNING:teuthology.provision.fog:timed out 2023-01-12T12:52:58.257 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-01-12T12:52:58.510 INFO:teuthology.provision.fog.smithi064:Node is ready 2023-01-12T12:52:58.569 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064.front.sepia.ceph.com 2023-01-12T12:52:58.628 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64 smithi064.front.sepia.ceph.com smithi064 2023-01-12T12:52:58.984 INFO:teuthology.provision.fog.smithi064:Deploy complete! 2023-01-12T12:52:59.192 INFO:teuthology.lock.ops:Checking smithi016.front.sepia.ceph.com 2023-01-12T12:52:59.193 INFO:teuthology.lock.ops:New key found. Updating... 2023-01-12T12:52:59.211 INFO:teuthology.lock.ops:Checking smithi064.front.sepia.ceph.com 2023-01-12T12:52:59.212 INFO:teuthology.lock.ops:New key found. Updating... 2023-01-12T12:52:59.229 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: set os type and version on server 2023-01-12T12:52:59.230 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-01-12T12:52:59.446 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2023-01-12T12:52:59.506 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2023-01-12T12:52:59.506 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8" 2023-01-12T12:52:59.507 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2023-01-12T12:52:59.508 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2023-01-12T12:52:59.508 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8" 2023-01-12T12:52:59.508 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2023-01-12T12:52:59.508 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 8" 2023-01-12T12:52:59.508 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2023-01-12T12:52:59.509 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-01-12T12:52:59.509 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2023-01-12T12:52:59.509 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-12T12:52:59.509 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-12T12:52:59.509 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-01-12T12:52:59.511 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-01-12T12:52:59.527 INFO:teuthology.lock.ops:Updating [smithi064.front.sepia.ceph.com]: set os type and version on server 2023-01-12T12:52:59.527 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-01-12T12:52:59.722 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2023-01-12T12:52:59.781 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2023-01-12T12:52:59.781 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="8" 2023-01-12T12:52:59.782 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2023-01-12T12:52:59.782 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2023-01-12T12:52:59.782 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="8" 2023-01-12T12:52:59.782 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el8" 2023-01-12T12:52:59.782 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 8" 2023-01-12T12:52:59.783 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2023-01-12T12:52:59.783 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-01-12T12:52:59.783 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2023-01-12T12:52:59.783 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-12T12:52:59.783 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-12T12:52:59.784 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-01-12T12:52:59.785 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2023-01-12T12:52:59.871 INFO:teuthology.dispatcher.supervisor:Running job 7135288 2023-01-12T12:52:59.871 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2023-01-12_12:11:23-orch:cephadm-wip-adk-testing-2023-01-11-2322-distro-default-smithi/7135288 --name adking-2023-01-12_12:11:23-orch:cephadm-wip-adk-testing-2023-01-11-2322-distro-default-smithi --description orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} -- /home/teuthworker/archive/adking-2023-01-12_12:11:23-orch:cephadm-wip-adk-testing-2023-01-11-2322-distro-default-smithi/7135288/orig.config.yaml 2023-01-12T12:52:59.876 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2023-01-12_12:11:23-orch:cephadm-wip-adk-testing-2023-01-11-2322-distro-default-smithi/7135288 2023-01-12T12:52:59.877 INFO:teuthology.dispatcher.supervisor:Job PID: 20211 2023-01-12T12:52:59.877 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-01-13T00:54:52.989 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2023-01-13T00:54:53.148 INFO:teuthology.kill:Killing Pids: {20211} 2023-01-13T00:54:54.000 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['host.a', 'client.0'] 2023-01-13T00:54:54.001 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['host.b', 'client.1'] 2023-01-13T00:54:54.002 INFO:teuthology.misc:Compressing logs... 2023-01-13T00:54:54.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-01-13T00:54:54.205 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-01-13T00:54:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2023-01-13T00:54:54.431 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2023-01-13T00:54:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2023-01-13T00:54:54.473 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2023-01-13T00:54:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2023-01-13T00:54:54.516 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2023-01-13T00:54:54.517 INFO:teuthology.misc:Compressing logs... 2023-01-13T00:54:54.796 INFO:teuthology.misc:Compressing logs... 2023-01-13T00:55:08.950 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/4bc1294c-9279-11ed-821b-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2023-01-13T00:55:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/4bc1294c-9279-11ed-821b-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2023-01-13T00:55:16.637 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/4bc1294c-9279-11ed-821b-001a4aab830c/ceph-mon.smithi064.log: file size changed while zipping 2023-01-13T00:55:23.936 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/4bc1294c-9279-11ed-821b-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2023-01-13T00:55:35.766 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/4bc1294c-9279-11ed-821b-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2023-01-13T00:55:40.069 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/4bc1294c-9279-11ed-821b-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2023-01-13T00:55:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/4bc1294c-9279-11ed-821b-001a4aab830c/ceph-mgr.smithi016.pchvvj.log: file size changed while zipping 2023-01-13T00:55:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/4bc1294c-9279-11ed-821b-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2023-01-13T00:56:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/4bc1294c-9279-11ed-821b-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2023-01-13T00:56:09.633 INFO:teuthology.orchestra.run.smithi064.stderr:gzip: /var/log/ceph/4bc1294c-9279-11ed-821b-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2023-01-13T00:56:23.810 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/4bc1294c-9279-11ed-821b-001a4aab830c/ceph-mon.smithi016.log: file size changed while zipping 2023-01-13T00:56:57.738 INFO:teuthology.kill:No teuthology processes running 2023-01-13T00:56:58.556 INFO:teuthology.kill:Nuking machines: ['smithi016', 'smithi064'] 2023-01-13T00:56:59.200 INFO:teuthology.kill:2023-01-13 00:56:59,200.200 INFO:teuthology.nuke:targets: 2023-01-13T00:56:59.201 INFO:teuthology.kill: smithi016.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbkhmu/yAMmdByjEwnQGjNUFBTXEEP60U5kKC9tbd3pq/1ulS0bq0WtzvQSLSIvtpTVtTvZJTbd7RgpqNMuuwTb23RZfDIdPZDGnoMkfp21gKpe8m7aVAa+e98bPwlfAMw1iF5VCeJO4UIIF2vA9sxnGLUiCKNNXBgNvYQwiRgOTQV6yngUCFegs/pt1UHplTcY/0SEAD0pxPFQQV2V72Oxz6R0QO2oIx2+NKaC+O+MSUEcw5VKtBsEQBbpLH/BLHRT47QGhNbEAS/sbhzkaniFttoyxN/dObVms2du4TxuWyy7M7iLyQiK2+1yLwK2v0spPC2YyG4LNYspbi2HX8zVdC85ZbvrqTCU2duwOCU/TzMHkbCcYa352HQR6k0Un8ZfIDhl0qq21AQJTx07TDsvURDY1s1b1gZl61U6kJ/jtSHwwKXKCjxL8wo+1U2SbqeK1scJrZk7hbnnSLz5ipcuMzV9x856zTE/yInZ8N+ESuUQ1Y7+FiG22dLn9acmhc= 2023-01-13T00:56:59.201 INFO:teuthology.kill: smithi064.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC++0o9GnuyFxndVs7rNh0GTDKwk4HH1LPw/OFJk5UkqFLisZGpkq44Ns6+MJ2ugDa1fjPemWNPC9yWBVSgo0H1hZ3ssJXKkRY8mb1mUZZSOJ1vWJvrY30UdPIQgMAuzNU2CivuUCBcU7canJJi0Ej/VW2AFCiGPwObL5EbkPEhe7p6b2Y90G54FxAalMcD4Pazuxx2B8Rnivj9a2xYpco8yYiiizB8Y92h2DjPr2wnXR8TxSbwfFN66rfa0ZIbbA5+u59LGmpuV30IHJlspQnla3IC6a7TXrnHt3HPaTrmCQyUgCNnKfancddtSYLc0oJD6albCa9f34+Aqh9zsn6Iy8ihZF/k0p3ELqgcyioFM3xoohEZYZfYBBujflJcr+1HlOrTzovJ2/47jbMiOKJLWg6DFztKthl86Ph//3i/t19aLVbVs1FsNrGXek+CMw8u5uDoTcgvt3JauCn0gfP6TlKWNJeR+H9cenFF9Uj84thN9msnfbYrQoupqwCY4u0= 2023-01-13T00:56:59.202 INFO:teuthology.kill:2023-01-13 00:56:59,200.200 INFO:teuthology.nuke:Checking targets against current locks 2023-01-13T00:56:59.248 INFO:teuthology.kill:2023-01-13 00:56:59,248.248 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-13T00:56:59.278 INFO:teuthology.kill:2023-01-13 00:56:59,278.278 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-13T00:56:59.328 INFO:teuthology.kill:2023-01-13 00:56:59,327.327 INFO:teuthology.orchestra.console:Power off smithi016 2023-01-13T00:56:59.377 INFO:teuthology.kill:2023-01-13 00:56:59,376.376 INFO:teuthology.orchestra.console:Power off smithi064 2023-01-13T00:57:11.828 INFO:teuthology.kill:2023-01-13 00:57:11,827.827 INFO:teuthology.orchestra.console:Power off for smithi016 completed 2023-01-13T00:57:11.857 INFO:teuthology.kill:2023-01-13 00:57:11,857.857 INFO:teuthology.orchestra.console:Power off for smithi064 completed 2023-01-13T00:57:11.988 INFO:teuthology.kill:2023-01-13 00:57:11,988.988 INFO:teuthology.lock.ops:unlocked: smithi016.front.sepia.ceph.com 2023-01-13T00:57:12.000 INFO:teuthology.kill:2023-01-13 00:57:12,000.000 INFO:teuthology.lock.ops:unlocked: smithi064.front.sepia.ceph.com 2023-01-13T00:59:12.306 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2023-01-13T00:59:12.395 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi016 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-01-12_20:03:13-rados-pacific_16.2.11_RC6.6-distro-default-smithi/7138219 2023-01-13T00:59:12.395 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi064 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-01-12_20:03:13-rados-pacific_16.2.11_RC6.6-distro-default-smithi/7138219