2022-01-08T21:58:43.815 INFO:root:teuthology version: 1.1.0-00e09f05 2022-01-08T21:58:43.949 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi016.front.sepia.ceph.com' reimaging 2022-01-08T21:58:43.950 INFO:teuthology.lock.ops:Updating [ubuntu@smithi016.front.sepia.ceph.com]: reset os type and version on server 2022-01-08T21:58:43.951 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2022-01-08T21:58:43.974 INFO:teuthology.lock.ops:Node 'ubuntu@smithi016.front.sepia.ceph.com' reimaging is complete 2022-01-08T21:58:43.975 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi139.front.sepia.ceph.com' reimaging 2022-01-08T21:58:43.975 INFO:teuthology.lock.ops:Updating [ubuntu@smithi139.front.sepia.ceph.com]: reset os type and version on server 2022-01-08T21:58:43.975 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2022-01-08T21:58:44.001 INFO:teuthology.lock.ops:Node 'ubuntu@smithi139.front.sepia.ceph.com' reimaging is complete 2022-01-08T21:58:44.194 INFO:teuthology.provision.fog.smithi016:Scheduling deploy of rhel 8.4 2022-01-08T21:58:44.208 INFO:teuthology.provision.fog.smithi139:Scheduling deploy of rhel 8.4 2022-01-08T21:58:44.554 INFO:teuthology.orchestra.console:Power off smithi139 2022-01-08T21:58:44.566 INFO:teuthology.orchestra.console:Power off smithi016 2022-01-08T21:58:52.933 INFO:teuthology.orchestra.console:Power off for smithi016 completed 2022-01-08T21:58:53.035 INFO:teuthology.orchestra.console:Power on smithi016 2022-01-08T21:58:57.037 INFO:teuthology.orchestra.console:Power off for smithi139 completed 2022-01-08T21:58:57.139 INFO:teuthology.orchestra.console:Power on smithi139 2022-01-08T21:59:01.403 INFO:teuthology.orchestra.console:Power on for smithi016 completed 2022-01-08T21:59:01.504 INFO:teuthology.provision.fog.smithi016:Waiting for deploy to finish 2022-01-08T21:59:05.509 INFO:teuthology.orchestra.console:Power on for smithi139 completed 2022-01-08T21:59:05.611 INFO:teuthology.provision.fog.smithi139:Waiting for deploy to finish 2022-01-08T22:01:17.882 ERROR:teuthology.orchestra.connection:Error authenticating with smithi016.front.sepia.ceph.com: Authentication failed. 2022-01-08T22:03:34.177 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-08T22:03:34.405 INFO:teuthology.provision.fog.smithi016:Node is ready 2022-01-08T22:03:34.421 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016.front.sepia.ceph.com 2022-01-08T22:03:34.477 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16 smithi016.front.sepia.ceph.com smithi016 2022-01-08T22:03:34.803 INFO:teuthology.provision.fog.smithi016:Deploy complete! 2022-01-08T22:04:10.558 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-08T22:04:10.764 INFO:teuthology.provision.fog.smithi139:Node is ready 2022-01-08T22:04:10.779 INFO:teuthology.orchestra.run.smithi139.stdout:smithi139.front.sepia.ceph.com 2022-01-08T22:04:10.835 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139 smithi139.front.sepia.ceph.com smithi139 2022-01-08T22:04:11.157 INFO:teuthology.provision.fog.smithi139:Deploy complete! 2022-01-08T22:04:11.303 INFO:teuthology.lock.ops:Checking smithi016.front.sepia.ceph.com 2022-01-08T22:04:11.304 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-08T22:04:11.324 INFO:teuthology.lock.ops:Checking smithi139.front.sepia.ceph.com 2022-01-08T22:04:11.324 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-08T22:04:11.342 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: set os type and version on server 2022-01-08T22:04:11.342 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2022-01-08T22:04:11.583 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2022-01-08T22:04:11.640 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux" 2022-01-08T22:04:11.640 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.4 (Ootpa)" 2022-01-08T22:04:11.641 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel" 2022-01-08T22:04:11.641 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora" 2022-01-08T22:04:11.641 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.4" 2022-01-08T22:04:11.641 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2022-01-08T22:04:11.642 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-08T22:04:11.642 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2022-01-08T22:04:11.642 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-08T22:04:11.642 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/" 2022-01-08T22:04:11.643 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-08T22:04:11.643 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-08T22:04:11.643 INFO:teuthology.orchestra.run.smithi016.stdout: 2022-01-08T22:04:11.643 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-08T22:04:11.644 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-08T22:04:11.644 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-08T22:04:11.644 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-08T22:04:11.645 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2022-01-08T22:04:11.665 INFO:teuthology.lock.ops:Updating [smithi139.front.sepia.ceph.com]: set os type and version on server 2022-01-08T22:04:11.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2022-01-08T22:04:11.930 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2022-01-08T22:04:11.985 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="Red Hat Enterprise Linux" 2022-01-08T22:04:11.985 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="8.4 (Ootpa)" 2022-01-08T22:04:11.986 INFO:teuthology.orchestra.run.smithi139.stdout:ID="rhel" 2022-01-08T22:04:11.986 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="fedora" 2022-01-08T22:04:11.986 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="8.4" 2022-01-08T22:04:11.987 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el8" 2022-01-08T22:04:11.987 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-01-08T22:04:11.987 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2022-01-08T22:04:11.988 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-01-08T22:04:11.988 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://www.redhat.com/" 2022-01-08T22:04:11.989 INFO:teuthology.orchestra.run.smithi139.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-01-08T22:04:11.989 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-08T22:04:11.989 INFO:teuthology.orchestra.run.smithi139.stdout: 2022-01-08T22:04:11.990 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-08T22:04:11.990 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-01-08T22:04:11.990 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-01-08T22:04:11.991 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-01-08T22:04:11.992 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2022-01-08T22:04:12.090 INFO:teuthology.dispatcher.supervisor:Running job 6603487 2022-01-08T22:04:12.090 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_00e09f05e9fb2ab7c7f497f5e446613531743ae2/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-08_17:57:43-rados-wip-yuri8-testing-2022-01-07-1541-distro-default-smithi/6603487 --name yuriw-2022-01-08_17:57:43-rados-wip-yuri8-testing-2022-01-07-1541-distro-default-smithi --description rados/cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} -- /home/teuthworker/archive/yuriw-2022-01-08_17:57:43-rados-wip-yuri8-testing-2022-01-07-1541-distro-default-smithi/6603487/orig.config.yaml 2022-01-08T22:04:12.094 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-08_17:57:43-rados-wip-yuri8-testing-2022-01-07-1541-distro-default-smithi/6603487 2022-01-08T22:04:12.095 INFO:teuthology.dispatcher.supervisor:Job PID: 17887 2022-01-08T22:04:12.095 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-09T10:04:32.131 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-01-09T10:04:32.282 INFO:teuthology.kill:Killing Pids: {17887} 2022-01-09T10:04:33.500 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['host.a', 'client.0'] 2022-01-09T10:04:33.501 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['host.b', 'client.1'] 2022-01-09T10:04:33.501 INFO:teuthology.misc:Compressing logs... 2022-01-09T10:04:33.502 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2022-01-09T10:04:33.697 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2022-01-09T10:04:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-09T10:04:33.969 INFO:teuthology.orchestra.run.smithi139.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-09T10:04:34.013 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-09T10:04:34.013 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2022-01-09T10:04:34.077 INFO:teuthology.orchestra.run.smithi139.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-09T10:04:34.077 INFO:teuthology.orchestra.run.smithi139.stderr:tar: Error is not recoverable: exiting now 2022-01-09T10:04:34.079 INFO:teuthology.misc:Compressing logs... 2022-01-09T10:04:34.284 INFO:teuthology.misc:Compressing logs... 2022-01-09T10:04:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/7127d2ec-70d0-11ec-8c32-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-09T10:04:44.186 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/7127d2ec-70d0-11ec-8c32-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-09T10:04:44.252 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/7127d2ec-70d0-11ec-8c32-001a4aab830c/ceph-mgr.smithi139.whvxyw.log: file size changed while zipping 2022-01-09T10:04:48.036 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/7127d2ec-70d0-11ec-8c32-001a4aab830c/ceph-mon.smithi139.log: file size changed while zipping 2022-01-09T10:04:53.760 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/7127d2ec-70d0-11ec-8c32-001a4aab830c/ceph-mon.smithi016.log: file size changed while zipping 2022-01-09T10:04:55.655 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/7127d2ec-70d0-11ec-8c32-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-09T10:05:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/7127d2ec-70d0-11ec-8c32-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-09T10:05:05.338 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/7127d2ec-70d0-11ec-8c32-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-09T10:05:11.893 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/7127d2ec-70d0-11ec-8c32-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-09T10:05:13.375 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/7127d2ec-70d0-11ec-8c32-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-09T10:05:23.326 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/7127d2ec-70d0-11ec-8c32-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-09T10:05:25.322 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/7127d2ec-70d0-11ec-8c32-001a4aab830c/ceph-mgr.smithi016.epgrsi.log: file size changed while zipping 2022-01-09T10:05:57.082 INFO:teuthology.kill:No teuthology processes running 2022-01-09T10:05:58.180 INFO:teuthology.kill:Nuking machines: ['smithi016', 'smithi139'] 2022-01-09T10:05:59.106 INFO:teuthology.kill:2022-01-09 10:05:59,106.106 INFO:teuthology.nuke:targets: 2022-01-09T10:05:59.107 INFO:teuthology.kill: smithi016.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGBUvgEhISkG0ETNBS7I5Uomi21rMBu8m+V5+slA0DSntPL4FM6MCXGMlvFxOMzWz4MVkoIPZlE8hr11N4kZxUaYwjIp7BFhsHfc4uWxSc+7xpmQfcZ3vRDaKlM+NRV742mTw6tyrR8oD9TAEmp+T5EyS14C9UJXhJ8ymY7IbArczp3pvXM5I5KZ0oCojPJgaMYDr+LJlQ7r23C45GiLAWknvTjFFYqCtYUcVPVbtAixR5NIa5YNqu0FDl7SFvzTSC5k6wDbIaJFvwbUUuTYQrzExklLFrj4ZGaOMmDCNVLYyt6/hVj7MkkYe5jVUNAHVcaNOKp0LhoKmsrBG+RvAQ01MZ1jHpb502ZdKrPhltVsxPYNiB935t6MP1nCjdLWQAKnlUh6sGgdad2jNz3QbwdJkjMrqt5dkfRzZnYkNNPebw66TG9MNxNYYGlRrvTl6FCm8sl8eyJAjgrQZ03swE4c5WLz7iY1bVZi6FN8V11MKJ2XnE3kmsJC9562PBhyE= 2022-01-09T10:05:59.107 INFO:teuthology.kill: smithi139.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLza+HVx80tiWBwFoPpGq0oKhtzFtSedBD+0D1P4n+ry9CxZ8SXYQjGr5jZn4Yn39PH5aWzrbOvdWj430Fs62iCIfOAn9rQSyXleLZx0A5wd4pK2bIIaAphfUc/rao29iLG0GoCKuXAM/Q6dSx5TmaNypuLEa0tWBlaFMwK04r0HBwu/521h8N/YxGmKUxG5iFKcDE3XLHUdFRYRLv0/ORjY7FZVX0dzLkrdmihJzJC6xwp1hdMP0DPfWszfe7sISUXsJuPhAKAIPL/KPRKbPU8wZzRgvDUMo5Tf+zKeuhEwxAtPOm8m1HSIXGTVUT7uv+Z65qZ3vMFJuuoGS1Pr2F7DV055ItSl8166FU+ouKBQWrqKfE7klyfpLozDORs9n6JpbN85kSTQhyLsQaPnJU70qBAEZTnaGE0SyS711xPCfdu+8pB/v7qFehXXBdSh7wG4EDv9JqPywQx2r4v4961RGKGQfLQ3kF5Ul6jAqe8ByFEkwRvZVWDXO18pxSPj8= 2022-01-09T10:05:59.134 INFO:teuthology.kill:2022-01-09 10:05:59,134.134 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-09T10:05:59.137 INFO:teuthology.kill:2022-01-09 10:05:59,137.137 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-09T10:05:59.214 INFO:teuthology.kill:2022-01-09 10:05:59,214.214 INFO:teuthology.orchestra.console:Power off smithi139 2022-01-09T10:05:59.228 INFO:teuthology.kill:2022-01-09 10:05:59,227.227 INFO:teuthology.orchestra.console:Power off smithi016 2022-01-09T10:06:07.593 INFO:teuthology.kill:2022-01-09 10:06:07,592.592 INFO:teuthology.orchestra.console:Power off for smithi016 completed 2022-01-09T10:06:07.737 INFO:teuthology.kill:2022-01-09 10:06:07,736.736 INFO:teuthology.lock.ops:unlocked: smithi016.front.sepia.ceph.com 2022-01-09T10:06:11.699 INFO:teuthology.kill:2022-01-09 10:06:11,698.698 INFO:teuthology.orchestra.console:Power off for smithi139 completed 2022-01-09T10:06:11.843 INFO:teuthology.kill:2022-01-09 10:06:11,843.843 INFO:teuthology.lock.ops:unlocked: smithi139.front.sepia.ceph.com 2022-01-09T10:08:12.041 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-09T10:08:12.132 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi016 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-08_14:23:02-upgrade:nautilus-x-pacific-distro-default-smithi/6602443 2022-01-09T10:08:12.133 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi139 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-08_14:23:02-upgrade:nautilus-x-pacific-distro-default-smithi/6602443