2022-02-08T21:54:21.982 INFO:root:teuthology version: 1.1.0-7cebb3f2
2022-02-08T21:54:22.218 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi087.front.sepia.ceph.com' reimaging
2022-02-08T21:54:22.220 INFO:teuthology.lock.ops:Updating [ubuntu@smithi087.front.sepia.ceph.com]: reset os type and version on server
2022-02-08T21:54:22.220 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server
2022-02-08T21:54:22.331 INFO:teuthology.lock.ops:Node 'ubuntu@smithi087.front.sepia.ceph.com' reimaging is complete
2022-02-08T21:54:22.332 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi097.front.sepia.ceph.com' reimaging
2022-02-08T21:54:22.332 INFO:teuthology.lock.ops:Updating [ubuntu@smithi097.front.sepia.ceph.com]: reset os type and version on server
2022-02-08T21:54:22.332 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server
2022-02-08T21:54:22.357 INFO:teuthology.lock.ops:Node 'ubuntu@smithi097.front.sepia.ceph.com' reimaging is complete
2022-02-08T21:54:22.547 INFO:teuthology.provision.fog.smithi087:Scheduling deploy of centos 8.stream
2022-02-08T21:54:22.570 INFO:teuthology.provision.fog.smithi097:Scheduling deploy of centos 8.stream
2022-02-08T21:54:22.935 INFO:teuthology.orchestra.console:Power off smithi087
2022-02-08T21:54:22.975 INFO:teuthology.orchestra.console:Power off smithi097
2022-02-08T21:54:35.427 INFO:teuthology.orchestra.console:Power off for smithi087 completed
2022-02-08T21:54:35.455 INFO:teuthology.orchestra.console:Power off for smithi097 completed
2022-02-08T21:54:35.528 INFO:teuthology.orchestra.console:Power on smithi087
2022-02-08T21:54:35.557 INFO:teuthology.orchestra.console:Power on smithi097
2022-02-08T21:54:43.895 INFO:teuthology.orchestra.console:Power on for smithi087 completed
2022-02-08T21:54:43.918 INFO:teuthology.orchestra.console:Power on for smithi097 completed
2022-02-08T21:54:43.997 INFO:teuthology.provision.fog.smithi087:Waiting for deploy to finish
2022-02-08T21:54:44.020 INFO:teuthology.provision.fog.smithi097:Waiting for deploy to finish
2022-02-08T21:58:51.207 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-08T21:59:21.017 INFO:teuthology.provision.fog.smithi097:Node is ready
2022-02-08T21:59:21.041 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097.front.sepia.ceph.com
2022-02-08T21:59:21.100 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97 smithi097.front.sepia.ceph.com smithi097
2022-02-08T21:59:21.458 INFO:teuthology.provision.fog.smithi097:Deploy complete!
2022-02-08T21:59:25.347 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-08T21:59:25.612 INFO:teuthology.provision.fog.smithi087:Node is ready
2022-02-08T21:59:25.629 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087.front.sepia.ceph.com
2022-02-08T21:59:25.687 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87 smithi087.front.sepia.ceph.com smithi087
2022-02-08T21:59:26.036 INFO:teuthology.provision.fog.smithi087:Deploy complete!
2022-02-08T21:59:26.215 INFO:teuthology.lock.ops:Checking smithi087.front.sepia.ceph.com
2022-02-08T21:59:26.216 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-08T21:59:26.238 INFO:teuthology.lock.ops:Checking smithi097.front.sepia.ceph.com
2022-02-08T21:59:26.238 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-08T21:59:26.257 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: set os type and version on server
2022-02-08T21:59:26.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com'
2022-02-08T21:59:26.446 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64
2022-02-08T21:59:26.503 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="CentOS Stream"
2022-02-08T21:59:26.504 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="8"
2022-02-08T21:59:26.504 INFO:teuthology.orchestra.run.smithi087.stdout:ID="centos"
2022-02-08T21:59:26.504 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="rhel fedora"
2022-02-08T21:59:26.504 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="8"
2022-02-08T21:59:26.505 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el8"
2022-02-08T21:59:26.505 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-08T21:59:26.505 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31"
2022-02-08T21:59:26.505 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-08T21:59:26.506 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://centos.org/"
2022-02-08T21:59:26.506 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-08T21:59:26.506 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-08T21:59:26.506 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-08T21:59:26.508 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server
2022-02-08T21:59:26.526 INFO:teuthology.lock.ops:Updating [smithi097.front.sepia.ceph.com]: set os type and version on server
2022-02-08T21:59:26.527 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com'
2022-02-08T21:59:26.726 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64
2022-02-08T21:59:26.786 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream"
2022-02-08T21:59:26.787 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="8"
2022-02-08T21:59:26.787 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos"
2022-02-08T21:59:26.787 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora"
2022-02-08T21:59:26.787 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="8"
2022-02-08T21:59:26.788 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el8"
2022-02-08T21:59:26.788 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-08T21:59:26.788 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31"
2022-02-08T21:59:26.789 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-08T21:59:26.789 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/"
2022-02-08T21:59:26.790 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-08T21:59:26.790 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-08T21:59:26.791 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-08T21:59:26.791 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server
2022-02-08T21:59:26.896 INFO:teuthology.dispatcher.supervisor:Running job 6670642
2022-02-08T21:59:26.896 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-02-08_17:00:23-rados-wip-yuri5-testing-2022-02-08-0733-pacific-distro-default-smithi/6670642 --name yuriw-2022-02-08_17:00:23-rados-wip-yuri5-testing-2022-02-08-0733-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2022-02-08_17:00:23-rados-wip-yuri5-testing-2022-02-08-0733-pacific-distro-default-smithi/6670642/orig.config.yaml
2022-02-08T21:59:26.899 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-02-08_17:00:23-rados-wip-yuri5-testing-2022-02-08-0733-pacific-distro-default-smithi/6670642
2022-02-08T21:59:26.900 INFO:teuthology.dispatcher.supervisor:Job PID: 29191
2022-02-08T21:59:26.900 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-09T04:29:56.383 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-09T04:29:56.576 INFO:teuthology.kill:Killing Pids: {29191}
2022-02-09T04:29:57.753 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-02-09T04:29:57.754 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-02-09T04:29:57.754 INFO:teuthology.misc:Compressing logs...
2022-02-09T04:29:57.755 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com'
2022-02-09T04:29:57.951 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com'
2022-02-09T04:29:58.000 INFO:teuthology.orchestra.run.smithi087.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-09T04:29:58.187 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-09T04:29:58.229 INFO:teuthology.orchestra.run.smithi087.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-09T04:29:58.230 INFO:teuthology.orchestra.run.smithi087.stderr:tar: Error is not recoverable: exiting now
2022-02-09T04:29:58.275 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-09T04:29:58.276 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now
2022-02-09T04:29:58.277 INFO:teuthology.misc:Compressing logs...
2022-02-09T04:29:58.577 INFO:teuthology.misc:Compressing logs...
2022-02-09T04:29:59.029 INFO:teuthology.orchestra.run.smithi097.stderr:gzip: /var/log/ceph/d2536690-892a-11ec-8c35-001a4aab830c/ceph-mon.smithi097.log: file size changed while zipping
2022-02-09T04:30:00.168 INFO:teuthology.orchestra.run.smithi097.stderr:gzip: /var/log/ceph/d2536690-892a-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-02-09T04:30:01.033 INFO:teuthology.orchestra.run.smithi087.stderr:gzip: /var/log/ceph/d2536690-892a-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-02-09T04:30:01.375 INFO:teuthology.orchestra.run.smithi097.stderr:gzip: /var/log/ceph/d2536690-892a-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-02-09T04:30:02.399 INFO:teuthology.orchestra.run.smithi097.stderr:gzip: /var/log/ceph/d2536690-892a-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-02-09T04:30:02.470 INFO:teuthology.orchestra.run.smithi087.stderr:gzip: /var/log/ceph/d2536690-892a-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-02-09T04:30:03.101 INFO:teuthology.orchestra.run.smithi087.stderr:gzip: /var/log/ceph/d2536690-892a-11ec-8c35-001a4aab830c/ceph-mgr.smithi087.wfipnv.log: file size changed while zipping
2022-02-09T04:30:03.575 INFO:teuthology.orchestra.run.smithi097.stderr:gzip: /var/log/ceph/d2536690-892a-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-02-09T04:30:05.444 INFO:teuthology.orchestra.run.smithi087.stderr:gzip: /var/log/ceph/d2536690-892a-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-02-09T04:30:07.675 INFO:teuthology.orchestra.run.smithi087.stderr:gzip: /var/log/ceph/d2536690-892a-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-02-09T04:30:09.933 INFO:teuthology.orchestra.run.smithi087.stderr:gzip: /var/log/ceph/d2536690-892a-11ec-8c35-001a4aab830c/ceph-mon.smithi087.log: file size changed while zipping
2022-02-09T04:30:29.134 INFO:teuthology.orchestra.run.smithi087.stderr:tar: .: file changed as we read it
2022-02-09T04:30:46.635 INFO:teuthology.kill:No teuthology processes running
2022-02-09T04:30:47.740 INFO:teuthology.kill:Nuking machines: ['smithi087', 'smithi097']
2022-02-09T04:30:48.721 INFO:teuthology.kill:2022-02-09 04:30:48,721.721 INFO:teuthology.nuke:targets:
2022-02-09T04:30:48.722 INFO:teuthology.kill:  smithi087.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD2qUW/1duzN8Hq5AEK+nlXK58bOvWtMIgSNuRBm8TYZ4FiR3VGGUNCaUgg2NzXLhLLZq0UftWAyNtadUnIjHuZ4fX+YnSifQZcC/8NpkyZjjkG1ZUj8E9fFVxnERyrXWK+0WTEvofc1bei1X7JC6VY8gmRd0oIvUrA8A0IdDkbutkTlP2v2tsDBdxntlW0P6ys9klp6IjXWFIV2paDbR9LElsK1o5bsoUjtDvNnw5fuhYpZjX8KfCK1o0gYGhbiYl5qlBuJOkXsN0zJ8yF/E7OHEvAm+B60iomCTPb51wZUd1XFJcdGuMJmuj/RIJY+tUevMz2KsnmE+nv4wY8FM/kiHSUFzGAvnGtCZs55itpMCoVIlwqX/cjh5ZQCtca5M4FvFH7QF3PdIR9aSGmmDtM4pj9Lg8x4fJzuPDwi5qVJci/DIpUNn0x5FqmM4xQ5/hRomPIPFvthhsaRIRig/1ViHCmxCTlVdoecAB9mnFr9+qUOfFLzgP8i0NEi2K8v90=
2022-02-09T04:30:48.723 INFO:teuthology.kill:  smithi097.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu1yCyuhCXLc3iyOxoJv1w9znDKZE9DcbGNoY0VOil4RafaRl6+g9jNHjz4ySdGBnYe7LXj4Q8V8PXG25i/9WV8P0Jsu80b9MoaL1gA0x8vHyy2RWS6nTJY2haXk955WaKsbkohdnz2exsZOLkpD7l5pvKuqH0iFuXxbdSdp6TBOnR4Fb+rsYqXMkQP0+PUU7cRCpuMnT+hBRpm3tvDT60JY1PZSguMxv4ypPJ/YFBgNZEuRPD9bhPidUo7n8TCMnHfgvtjrVSfmCkJ7VR9wE5cqo/5hwaKns/4auqDSP+5lTp+RN/tCPX/QC44IpWAgTz+Cq4C3go9/W8c9yECFQiC+ypSHSx0askb2j46EF90do8sBNQsOMh6l2wGFn76F44vZ921qSbtvo0H3v0+mwihcVuDQMM1YaCWG7hzAJyL961PkVlkJpIYB5wnOYPNc+VatKDKGVkxsMhJqQT/Bq2+tWiPEHjirkYC0oE+8fRasN5S8+m4adqht2S0bcDQd8=
2022-02-09T04:30:48.748 INFO:teuthology.kill:2022-02-09 04:30:48,748.748 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-09T04:30:48.751 INFO:teuthology.kill:2022-02-09 04:30:48,751.751 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-09T04:30:48.826 INFO:teuthology.kill:2022-02-09 04:30:48,825.825 INFO:teuthology.orchestra.console:Power off smithi087
2022-02-09T04:30:48.840 INFO:teuthology.kill:2022-02-09 04:30:48,839.839 INFO:teuthology.orchestra.console:Power off smithi097
2022-02-09T04:30:57.205 INFO:teuthology.kill:2022-02-09 04:30:57,204.204 INFO:teuthology.orchestra.console:Power off for smithi097 completed
2022-02-09T04:30:57.349 INFO:teuthology.kill:2022-02-09 04:30:57,349.349 INFO:teuthology.lock.ops:unlocked: smithi097.front.sepia.ceph.com
2022-02-09T04:31:01.308 INFO:teuthology.kill:2022-02-09 04:31:01,308.308 INFO:teuthology.orchestra.console:Power off for smithi087 completed
2022-02-09T04:31:01.453 INFO:teuthology.kill:2022-02-09 04:31:01,453.453 INFO:teuthology.lock.ops:unlocked: smithi087.front.sepia.ceph.com
2022-02-09T04:33:01.766 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-09T04:33:01.864 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi087 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-02-08_04:17:02-fs-pacific-distro-default-smithi/6668698
2022-02-09T04:33:01.865 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi097 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-02-08_04:17:02-fs-pacific-distro-default-smithi/6668692