2022-03-03T04:22:56.764 INFO:root:teuthology version: 1.1.0-eeaeee38
2022-03-03T04:22:56.889 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi057.front.sepia.ceph.com' reimaging
2022-03-03T04:22:56.890 INFO:teuthology.lock.ops:Updating [ubuntu@smithi057.front.sepia.ceph.com]: reset os type and version on server
2022-03-03T04:22:56.891 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server
2022-03-03T04:22:56.916 INFO:teuthology.lock.ops:Node 'ubuntu@smithi057.front.sepia.ceph.com' reimaging is complete
2022-03-03T04:22:56.916 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging
2022-03-03T04:22:56.917 INFO:teuthology.lock.ops:Updating [ubuntu@smithi146.front.sepia.ceph.com]: reset os type and version on server
2022-03-03T04:22:56.917 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2022-03-03T04:22:56.936 INFO:teuthology.lock.ops:Node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging is complete
2022-03-03T04:22:57.132 INFO:teuthology.provision.fog.smithi057:Scheduling deploy of centos 8.stream
2022-03-03T04:22:57.138 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of centos 8.stream
2022-03-03T04:22:57.440 INFO:teuthology.orchestra.console:Power off smithi057
2022-03-03T04:22:57.488 INFO:teuthology.orchestra.console:Power off smithi146
2022-03-03T04:23:05.825 INFO:teuthology.orchestra.console:Power off for smithi057 completed
2022-03-03T04:23:05.926 INFO:teuthology.orchestra.console:Power on smithi057
2022-03-03T04:23:09.960 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2022-03-03T04:23:10.062 INFO:teuthology.orchestra.console:Power on smithi146
2022-03-03T04:23:14.296 INFO:teuthology.orchestra.console:Power on for smithi057 completed
2022-03-03T04:23:14.319 INFO:teuthology.orchestra.console:Power on for smithi146 completed
2022-03-03T04:23:14.397 INFO:teuthology.provision.fog.smithi057:Waiting for deploy to finish
2022-03-03T04:23:14.420 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish
2022-03-03T04:27:45.030 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-03T04:27:46.204 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-03T04:27:46.457 INFO:teuthology.provision.fog.smithi057:Node is ready
2022-03-03T04:27:46.476 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057.front.sepia.ceph.com
2022-03-03T04:27:46.534 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57 smithi057.front.sepia.ceph.com smithi057
2022-03-03T04:27:46.878 INFO:teuthology.provision.fog.smithi057:Deploy complete!
2022-03-03T04:28:09.806 INFO:teuthology.provision.fog.smithi146:Node is ready
2022-03-03T04:28:09.829 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com
2022-03-03T04:28:09.887 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146
2022-03-03T04:28:10.268 INFO:teuthology.provision.fog.smithi146:Deploy complete!
2022-03-03T04:28:10.417 INFO:teuthology.lock.ops:Checking smithi057.front.sepia.ceph.com
2022-03-03T04:28:10.418 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-03T04:28:10.438 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com
2022-03-03T04:28:10.439 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-03T04:28:10.459 INFO:teuthology.lock.ops:Updating [smithi057.front.sepia.ceph.com]: set os type and version on server
2022-03-03T04:28:10.459 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com'
2022-03-03T04:28:10.710 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64
2022-03-03T04:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="CentOS Stream"
2022-03-03T04:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="8"
2022-03-03T04:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout:ID="centos"
2022-03-03T04:28:10.769 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE="rhel fedora"
2022-03-03T04:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="8"
2022-03-03T04:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout:PLATFORM_ID="platform:el8"
2022-03-03T04:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="CentOS Stream 8"
2022-03-03T04:28:10.770 INFO:teuthology.orchestra.run.smithi057.stdout:ANSI_COLOR="0;31"
2022-03-03T04:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-03-03T04:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://centos.org/"
2022-03-03T04:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-03T04:28:10.771 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-03T04:28:10.772 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-03-03T04:28:10.773 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server
2022-03-03T04:28:10.793 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server
2022-03-03T04:28:10.793 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2022-03-03T04:28:10.989 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64
2022-03-03T04:28:11.048 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream"
2022-03-03T04:28:11.049 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8"
2022-03-03T04:28:11.049 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos"
2022-03-03T04:28:11.049 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora"
2022-03-03T04:28:11.050 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8"
2022-03-03T04:28:11.050 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8"
2022-03-03T04:28:11.050 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 8"
2022-03-03T04:28:11.050 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31"
2022-03-03T04:28:11.051 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-03-03T04:28:11.051 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/"
2022-03-03T04:28:11.051 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-03T04:28:11.051 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-03T04:28:11.051 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-03-03T04:28:11.052 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2022-03-03T04:28:11.143 INFO:teuthology.dispatcher.supervisor:Running job 6717315
2022-03-03T04:28:11.143 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_eeaeee38264fad8a01fd1ce912d8908676ed64bc/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2022-03-03_03:18:15-orch:cephadm-wip-adk2-testing-2022-03-02-1858-pacific-distro-basic-smithi/6717315 --name adking-2022-03-03_03:18:15-orch:cephadm-wip-adk2-testing-2022-03-02-1858-pacific-distro-basic-smithi --description orch:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/adking-2022-03-03_03:18:15-orch:cephadm-wip-adk2-testing-2022-03-02-1858-pacific-distro-basic-smithi/6717315/orig.config.yaml
2022-03-03T04:28:11.146 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2022-03-03_03:18:15-orch:cephadm-wip-adk2-testing-2022-03-02-1858-pacific-distro-basic-smithi/6717315
2022-03-03T04:28:11.147 INFO:teuthology.dispatcher.supervisor:Job PID: 5709
2022-03-03T04:28:11.148 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-03-03T10:58:40.100 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-03-03T10:58:40.287 INFO:teuthology.kill:Killing Pids: {5709}
2022-03-03T10:58:41.471 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-03-03T10:58:41.472 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-03-03T10:58:41.473 INFO:teuthology.misc:Compressing logs...
2022-03-03T10:58:41.474 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com'
2022-03-03T10:58:41.660 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2022-03-03T10:58:41.707 INFO:teuthology.orchestra.run.smithi057.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-03-03T10:58:41.895 INFO:teuthology.orchestra.run.smithi146.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-03-03T10:58:41.940 INFO:teuthology.orchestra.run.smithi057.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-03-03T10:58:41.940 INFO:teuthology.orchestra.run.smithi057.stderr:tar: Error is not recoverable: exiting now
2022-03-03T10:58:42.049 INFO:teuthology.orchestra.run.smithi146.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-03-03T10:58:42.050 INFO:teuthology.orchestra.run.smithi146.stderr:tar: Error is not recoverable: exiting now
2022-03-03T10:58:42.051 INFO:teuthology.misc:Compressing logs...
2022-03-03T10:58:42.340 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-03-03T10:58:42.499 INFO:teuthology.misc:Compressing logs...
2022-03-03T10:58:43.481 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/76e235ba-9aab-11ec-8c35-001a4aab830c/ceph-mgr.smithi146.jkfraw.log: file size changed while zipping
2022-03-03T10:58:44.105 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/76e235ba-9aab-11ec-8c35-001a4aab830c/ceph-mgr.smithi057.jutgvk.log: file size changed while zipping
2022-03-03T10:59:04.197 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/76e235ba-9aab-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-03-03T10:59:04.566 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/76e235ba-9aab-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-03-03T10:59:08.917 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/76e235ba-9aab-11ec-8c35-001a4aab830c/ceph-mon.smithi146.log: file size changed while zipping
2022-03-03T10:59:17.407 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/76e235ba-9aab-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-03-03T10:59:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/76e235ba-9aab-11ec-8c35-001a4aab830c/ceph-mon.smithi057.log: file size changed while zipping
2022-03-03T10:59:33.490 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/76e235ba-9aab-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-03-03T10:59:51.129 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/76e235ba-9aab-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-03-03T10:59:53.271 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/76e235ba-9aab-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-03-03T11:00:09.455 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/76e235ba-9aab-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-03-03T11:00:17.628 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/76e235ba-9aab-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-03-03T11:00:36.857 INFO:teuthology.orchestra.run.smithi057.stderr:tar: .: file changed as we read it
2022-03-03T11:00:55.134 INFO:teuthology.kill:No teuthology processes running
2022-03-03T11:00:56.302 INFO:teuthology.kill:Nuking machines: ['smithi057', 'smithi146']
2022-03-03T11:00:57.287 INFO:teuthology.kill:2022-03-03 11:00:57,286.286 INFO:teuthology.nuke:targets:
2022-03-03T11:00:57.287 INFO:teuthology.kill:  smithi057.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlPAPsi6XDrSoTxd7AC53j6jqzXRUItfnXde3O0EF6bV3BmJjHR1DmQ/XN+eN471HaonZ4WY2UX+/yoFZw2ymc74nhj/pQa76szqSZq3zjbRMDwYmhjwup42YJzF4h6Gr4VHY/WFjqhXTN9zm9E6rpwTs8sUfSlt4Zqmvq5CVRA7VrhtSY2iTKqBo7JBrq27RpoYn6h1DQX3X6A6QyhsSQ+11RxwISWKjCKP4S1dY1ulvxDmdVOcDFIy9bclIueb7Dt4jLtIi8J7DkJ5Z8aByCLsjKuApAonpgEdS+GmROYqx4R7jrTU21BtHSU2KvPwj3nkkdToM2I52jozfGJyp+C46zpjYKDHq8KrX4P7ZXfpZXldGvRzrtIgP+j7AlJyuFeBGedl6XmeKHav6S/pYQDbmYRQg85NN5l6KESt0jBRczCWGFyfneRu3DgeoZyLGERlDkdYQMvP2IUCI9Hxf9R1RFam44sYhXypOmY78iVkOV0k2iyzG0BWAXkuXcjM0=
2022-03-03T11:00:57.288 INFO:teuthology.kill:  smithi146.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYPDmi16P16YaMT6ktThpBrG8qeXceuN173tdx6CyDdLAYsaB/ZdlCZScPDtzxlHMbwiEpdtBblL4RIL8Ql665cCWl5C7wQhJJfpahYT2Lzt6Lj+ysy5ALuBF3bj3YvyZ4mXmVqHe/LfEc+xaZQc3bF98aPzQZrJaT60LBkpL8Xd7j61eZxwUQ9y61zZdQoNfe0QtQiUFSvzlfyinO67HnNY+v1tszEsnec0Cz3yd2oFnwxzt7JhJ8IkgtnHWZY+Nqa5qspcEK6wSahQYqRZjXfddxkDtOoJGsA1W1W73mRv0gQ2T7ygZYVOLCJ9CQd9R2MoF2haynOFKh2hcGaZfcpLU9raJeOvd5WtpSzm+7hZTfxV8O0pz+qgsOyl5nHUO4uEVCcp2rDppmbkESX+kkUVP3PbGliDb7B0YCV66I8dZ1Hxi9Cm0ZIpvzD/2LA7N+NyVlSz4Rx2U5SvbBsqaRzHj8cr5kT2mmkHbbftUH8JnAXTTCw+6hXWExhmUr15M=
2022-03-03T11:00:57.312 INFO:teuthology.kill:2022-03-03 11:00:57,312.312 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-03T11:00:57.315 INFO:teuthology.kill:2022-03-03 11:00:57,315.315 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-03T11:00:57.390 INFO:teuthology.kill:2022-03-03 11:00:57,390.390 INFO:teuthology.orchestra.console:Power off smithi146
2022-03-03T11:00:57.411 INFO:teuthology.kill:2022-03-03 11:00:57,410.410 INFO:teuthology.orchestra.console:Power off smithi057
2022-03-03T11:01:05.771 INFO:teuthology.kill:2022-03-03 11:01:05,770.770 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2022-03-03T11:01:05.920 INFO:teuthology.kill:2022-03-03 11:01:05,919.919 INFO:teuthology.lock.ops:unlocked: smithi146.front.sepia.ceph.com
2022-03-03T11:01:09.888 INFO:teuthology.kill:2022-03-03 11:01:09,888.888 INFO:teuthology.orchestra.console:Power off for smithi057 completed
2022-03-03T11:01:10.033 INFO:teuthology.kill:2022-03-03 11:01:10,033.033 INFO:teuthology.lock.ops:unlocked: smithi057.front.sepia.ceph.com
2022-03-03T11:03:10.365 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-03-03T11:03:10.463 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi057 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-03-03_05:07:02-rgw-pacific-distro-default-smithi/6717493
2022-03-03T11:03:10.464 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi146 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-03-03_05:07:02-rgw-pacific-distro-default-smithi/6717493