2022-01-11T21:00:23.814 INFO:root:teuthology version: 1.1.0-00e09f05 2022-01-11T21:00:23.957 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi082.front.sepia.ceph.com' reimaging 2022-01-11T21:00:23.958 INFO:teuthology.lock.ops:Updating [ubuntu@smithi082.front.sepia.ceph.com]: reset os type and version on server 2022-01-11T21:00:23.959 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2022-01-11T21:00:23.977 INFO:teuthology.lock.ops:Node 'ubuntu@smithi082.front.sepia.ceph.com' reimaging is complete 2022-01-11T21:00:23.977 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi145.front.sepia.ceph.com' reimaging 2022-01-11T21:00:23.978 INFO:teuthology.lock.ops:Updating [ubuntu@smithi145.front.sepia.ceph.com]: reset os type and version on server 2022-01-11T21:00:23.978 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2022-01-11T21:00:23.999 INFO:teuthology.lock.ops:Node 'ubuntu@smithi145.front.sepia.ceph.com' reimaging is complete 2022-01-11T21:00:24.198 INFO:teuthology.provision.fog.smithi082:Scheduling deploy of centos 8.stream 2022-01-11T21:00:24.208 INFO:teuthology.provision.fog.smithi145:Scheduling deploy of centos 8.stream 2022-01-11T21:00:24.546 INFO:teuthology.orchestra.console:Power off smithi145 2022-01-11T21:00:24.564 INFO:teuthology.orchestra.console:Power off smithi082 2022-01-11T21:00:32.920 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2022-01-11T21:00:33.022 INFO:teuthology.orchestra.console:Power on smithi145 2022-01-11T21:00:37.039 INFO:teuthology.orchestra.console:Power off for smithi082 completed 2022-01-11T21:00:37.141 INFO:teuthology.orchestra.console:Power on smithi082 2022-01-11T21:00:41.398 INFO:teuthology.orchestra.console:Power on for smithi082 completed 2022-01-11T21:00:41.472 INFO:teuthology.orchestra.console:Power on for smithi145 completed 2022-01-11T21:00:41.500 INFO:teuthology.provision.fog.smithi082:Waiting for deploy to finish 2022-01-11T21:00:41.574 INFO:teuthology.provision.fog.smithi145:Waiting for deploy to finish 2022-01-11T21:05:31.443 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-11T21:05:31.462 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-11T21:05:31.724 INFO:teuthology.provision.fog.smithi145:Node is ready 2022-01-11T21:05:31.728 INFO:teuthology.provision.fog.smithi082:Node is ready 2022-01-11T21:05:31.741 INFO:teuthology.orchestra.run.smithi145.stdout:smithi145.front.sepia.ceph.com 2022-01-11T21:05:31.746 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082.front.sepia.ceph.com 2022-01-11T21:05:31.799 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145 smithi145.front.sepia.ceph.com smithi145 2022-01-11T21:05:31.803 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82 smithi082.front.sepia.ceph.com smithi082 2022-01-11T21:05:32.143 INFO:teuthology.provision.fog.smithi082:Deploy complete! 2022-01-11T21:05:32.145 INFO:teuthology.provision.fog.smithi145:Deploy complete! 2022-01-11T21:05:32.286 INFO:teuthology.lock.ops:Checking smithi082.front.sepia.ceph.com 2022-01-11T21:05:32.287 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-11T21:05:32.305 INFO:teuthology.lock.ops:Checking smithi145.front.sepia.ceph.com 2022-01-11T21:05:32.305 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-11T21:05:32.323 INFO:teuthology.lock.ops:Updating [smithi082.front.sepia.ceph.com]: set os type and version on server 2022-01-11T21:05:32.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2022-01-11T21:05:32.509 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2022-01-11T21:05:32.563 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream" 2022-01-11T21:05:32.564 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8" 2022-01-11T21:05:32.564 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos" 2022-01-11T21:05:32.564 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora" 2022-01-11T21:05:32.565 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8" 2022-01-11T21:05:32.565 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8" 2022-01-11T21:05:32.565 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-11T21:05:32.565 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2022-01-11T21:05:32.565 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-11T21:05:32.566 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/" 2022-01-11T21:05:32.566 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-11T21:05:32.566 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-11T21:05:32.566 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-11T21:05:32.568 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2022-01-11T21:05:32.586 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: set os type and version on server 2022-01-11T21:05:32.586 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2022-01-11T21:05:32.769 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2022-01-11T21:05:32.825 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2022-01-11T21:05:32.825 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="8" 2022-01-11T21:05:32.825 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2022-01-11T21:05:32.825 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2022-01-11T21:05:32.826 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="8" 2022-01-11T21:05:32.826 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el8" 2022-01-11T21:05:32.826 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-11T21:05:32.826 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2022-01-11T21:05:32.827 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-11T21:05:32.827 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2022-01-11T21:05:32.827 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-11T21:05:32.828 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-11T21:05:32.828 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-11T21:05:32.828 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2022-01-11T21:05:32.929 INFO:teuthology.dispatcher.supervisor:Running job 6608555 2022-01-11T21:05:32.929 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_00e09f05e9fb2ab7c7f497f5e446613531743ae2/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-11_19:17:55-rados-wip-yuri5-testing-2022-01-11-0843-distro-default-smithi/6608555 --name yuriw-2022-01-11_19:17:55-rados-wip-yuri5-testing-2022-01-11-0843-distro-default-smithi --description rados/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} -- /home/teuthworker/archive/yuriw-2022-01-11_19:17:55-rados-wip-yuri5-testing-2022-01-11-0843-distro-default-smithi/6608555/orig.config.yaml 2022-01-11T21:05:32.934 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-11_19:17:55-rados-wip-yuri5-testing-2022-01-11-0843-distro-default-smithi/6608555 2022-01-11T21:05:32.935 INFO:teuthology.dispatcher.supervisor:Job PID: 27267 2022-01-11T21:05:32.935 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-12T09:06:27.319 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-01-12T09:06:27.473 INFO:teuthology.kill:Killing Pids: {27267} 2022-01-12T09:06:28.631 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-12T09:06:28.631 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-12T09:06:28.632 INFO:teuthology.misc:Compressing logs... 2022-01-12T09:06:28.632 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2022-01-12T09:06:28.820 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2022-01-12T09:06:28.849 INFO:teuthology.orchestra.run.smithi082.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-12T09:06:29.040 INFO:teuthology.orchestra.run.smithi145.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-12T09:06:29.082 INFO:teuthology.orchestra.run.smithi082.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-12T09:06:29.082 INFO:teuthology.orchestra.run.smithi082.stderr:tar: Error is not recoverable: exiting now 2022-01-12T09:06:29.121 INFO:teuthology.orchestra.run.smithi145.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-12T09:06:29.121 INFO:teuthology.orchestra.run.smithi145.stderr:tar: Error is not recoverable: exiting now 2022-01-12T09:06:29.122 INFO:teuthology.misc:Compressing logs... 2022-01-12T09:06:29.693 INFO:teuthology.misc:Compressing logs... 2022-01-12T09:06:31.463 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/5ff52ec0-7323-11ec-8c32-001a4aab830c/ceph-mgr.smithi082.mvdgsp.log: file size changed while zipping 2022-01-12T09:06:32.756 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /var/log/ceph/5ff52ec0-7323-11ec-8c32-001a4aab830c/ceph-mon.smithi145.log: file size changed while zipping 2022-01-12T09:06:39.402 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/5ff52ec0-7323-11ec-8c32-001a4aab830c/ceph-mon.smithi082.log: file size changed while zipping 2022-01-12T09:06:49.389 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /var/log/ceph/5ff52ec0-7323-11ec-8c32-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-12T09:06:50.962 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/5ff52ec0-7323-11ec-8c32-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-12T09:07:05.769 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /var/log/ceph/5ff52ec0-7323-11ec-8c32-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-12T09:07:07.978 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/5ff52ec0-7323-11ec-8c32-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-12T09:07:20.561 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /var/log/ceph/5ff52ec0-7323-11ec-8c32-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-12T09:07:24.682 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/5ff52ec0-7323-11ec-8c32-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-12T09:07:30.341 INFO:teuthology.orchestra.run.smithi145.stderr:gzip: /var/log/ceph/5ff52ec0-7323-11ec-8c32-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-12T09:07:39.514 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/5ff52ec0-7323-11ec-8c32-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-12T09:08:12.218 INFO:teuthology.orchestra.run.smithi082.stderr:tar: .: file changed as we read it 2022-01-12T09:08:39.072 INFO:teuthology.kill:No teuthology processes running 2022-01-12T09:08:40.239 INFO:teuthology.kill:Nuking machines: ['smithi082', 'smithi145'] 2022-01-12T09:08:41.261 INFO:teuthology.kill:2022-01-12 09:08:41,260.260 INFO:teuthology.nuke:targets: 2022-01-12T09:08:41.262 INFO:teuthology.kill: smithi082.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIf4StzHMTE4mgT50p/z9oq9bG0zUht4LMspRyQc6RkeQzmKC3HZhsfdDZYlDd9/k+6QH0l5CCJJSbNGnp3BIuDRXYjkOLmAJizDQ/SUqOrbCfx5boGoXb23oPoTenrjxzuRbIb2tUigcSB7PL1CiWfDAkRCbITTxp5dYs1/SV7B2yBFXR4nF8eUwzFurMGfP9dwK4oQ8/QhwToXkABoH+5qVf/7rygH7MfIapu821gHhr8I8/RE/0v9OjgDnkrB0n1i1CvfjkAzI9wxL2Tn1S4cuf4VZEApdc+lkrVRUqjxtbyeztBKdVhyUCbhbXWD8ud6dIKq5PLPuI/2gxnHw0icUHQMJelIejoCpqmcihj7yh4/bbhSczJcCXPo+n09FdXsafUcBX0y8ybodAb/jeWUxku+RvtFxZWH9AWofylWhDBvvbD/mcPL2Pi4wHqyD6FIP3jOaEex4q8OqeUTixCMlBI5eEbqacEgfkutd1kzISq57jqhePmmDkcKUD0m0= 2022-01-12T09:08:41.262 INFO:teuthology.kill: smithi145.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDK4uPuLoVCA/GknqSdiWtJnkSY5zlq2rkyVpBQ9l53ZlZ4tQpUTSPSziONy7CjLXx52B0CtZeEWV59a5oNVROQbnFD8hvGcFZsgBav0J1Obz7DIpMz06u2SI/OdK+PMY3GSuGi/Cu9zUo/VXq+TyP97Gznmc5pV3nFJB8Oa0zOG0OEZRfn8LPcYV09h6x+EmJ+tB3tCJKvsb5r35dVCwCkEQduE2RP9kA7LDqUXSaUGsvm2T7wMXbYdZMvfS8ZViBa7+ubpckjYNIwh+0af+toDkhgMp9UqgC39WwfvOfcZYoYZH8+3ezMR8/IqkP67vmh+VR7P6vwM91P5dGJXHyG9UPBr5Lbsgpdoy5ebrqPXGiSDMdgKafioCQ1DPFYDC7oT1cC0jlVO4jLcK0U0r0i8pzXr3cFqShtVdzHgZMhGnoAw0X6gJVvX6eEwTCwxSyD3dpGeDd0WHIkoG/zCYwPGNLRu2ViJcTHixAMGaA5pdc2/xiH1AgG1k2Ec00Ybf8= 2022-01-12T09:08:41.289 INFO:teuthology.kill:2022-01-12 09:08:41,289.289 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-12T09:08:41.293 INFO:teuthology.kill:2022-01-12 09:08:41,293.293 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-12T09:08:41.369 INFO:teuthology.kill:2022-01-12 09:08:41,368.368 INFO:teuthology.orchestra.console:Power off smithi145 2022-01-12T09:08:41.382 INFO:teuthology.kill:2022-01-12 09:08:41,382.382 INFO:teuthology.orchestra.console:Power off smithi082 2022-01-12T09:08:49.741 INFO:teuthology.kill:2022-01-12 09:08:49,741.741 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2022-01-12T09:08:49.887 INFO:teuthology.kill:2022-01-12 09:08:49,886.886 INFO:teuthology.lock.ops:unlocked: smithi145.front.sepia.ceph.com 2022-01-12T09:08:53.855 INFO:teuthology.kill:2022-01-12 09:08:53,855.855 INFO:teuthology.orchestra.console:Power off for smithi082 completed 2022-01-12T09:08:54.001 INFO:teuthology.kill:2022-01-12 09:08:54,001.001 INFO:teuthology.lock.ops:unlocked: smithi082.front.sepia.ceph.com 2022-01-12T09:10:54.295 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-12T09:10:54.390 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi082 but it was locked by another job: /home/teuthworker/archive/mchangir-2022-01-11_08:23:28-fs-wip-mchangir-mds-retry-backtrace-validation-distro-default-smithi/6607750 2022-01-12T09:10:54.391 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi145 but it was locked by another job: /home/teuthworker/archive/mchangir-2022-01-11_08:23:28-fs-wip-mchangir-mds-retry-backtrace-validation-distro-default-smithi/6607748