2022-03-18T06:26:15.055 INFO:root:teuthology version: 1.1.0-aa078dbe
2022-03-18T06:26:15.224 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi139.front.sepia.ceph.com' reimaging
2022-03-18T06:26:15.225 INFO:teuthology.lock.ops:Updating [ubuntu@smithi139.front.sepia.ceph.com]: reset os type and version on server
2022-03-18T06:26:15.225 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server
2022-03-18T06:26:15.252 INFO:teuthology.lock.ops:Node 'ubuntu@smithi139.front.sepia.ceph.com' reimaging is complete
2022-03-18T06:26:15.252 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging
2022-03-18T06:26:15.252 INFO:teuthology.lock.ops:Updating [ubuntu@smithi185.front.sepia.ceph.com]: reset os type and version on server
2022-03-18T06:26:15.253 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server
2022-03-18T06:26:15.273 INFO:teuthology.lock.ops:Node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging is complete
2022-03-18T06:26:15.464 INFO:teuthology.provision.fog.smithi185:Scheduling deploy of centos 8.stream
2022-03-18T06:26:15.468 INFO:teuthology.provision.fog.smithi139:Scheduling deploy of centos 8.stream
2022-03-18T06:26:15.855 INFO:teuthology.orchestra.console:Power off smithi139
2022-03-18T06:26:15.897 INFO:teuthology.orchestra.console:Power off smithi185
2022-03-18T06:26:24.242 INFO:teuthology.orchestra.console:Power off for smithi139 completed
2022-03-18T06:26:24.345 INFO:teuthology.orchestra.console:Power on smithi139
2022-03-18T06:26:28.383 INFO:teuthology.orchestra.console:Power off for smithi185 completed
2022-03-18T06:26:28.485 INFO:teuthology.orchestra.console:Power on smithi185
2022-03-18T06:26:32.738 INFO:teuthology.orchestra.console:Power on for smithi185 completed
2022-03-18T06:26:32.767 INFO:teuthology.orchestra.console:Power on for smithi139 completed
2022-03-18T06:26:32.840 INFO:teuthology.provision.fog.smithi185:Waiting for deploy to finish
2022-03-18T06:26:32.868 INFO:teuthology.provision.fog.smithi139:Waiting for deploy to finish
2022-03-18T06:31:04.906 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-18T06:31:05.197 INFO:teuthology.provision.fog.smithi139:Node is ready
2022-03-18T06:31:05.215 INFO:teuthology.orchestra.run.smithi139.stdout:smithi139.front.sepia.ceph.com
2022-03-18T06:31:05.273 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139 smithi139.front.sepia.ceph.com smithi139
2022-03-18T06:31:05.618 INFO:teuthology.provision.fog.smithi139:Deploy complete!
2022-03-18T06:31:19.896 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-18T06:31:20.142 INFO:teuthology.provision.fog.smithi185:Node is ready
2022-03-18T06:31:20.201 INFO:teuthology.orchestra.run.smithi185.stdout:smithi185.front.sepia.ceph.com
2022-03-18T06:31:20.259 INFO:teuthology.orchestra.run.smithi185.stdout:172.21.15.185 smithi185.front.sepia.ceph.com smithi185
2022-03-18T06:31:20.598 INFO:teuthology.provision.fog.smithi185:Deploy complete!
2022-03-18T06:31:20.782 INFO:teuthology.lock.ops:Checking smithi139.front.sepia.ceph.com
2022-03-18T06:31:20.783 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-18T06:31:20.803 INFO:teuthology.lock.ops:Checking smithi185.front.sepia.ceph.com
2022-03-18T06:31:20.803 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-18T06:31:20.822 INFO:teuthology.lock.ops:Updating [smithi139.front.sepia.ceph.com]: set os type and version on server
2022-03-18T06:31:20.823 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com'
2022-03-18T06:31:21.071 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64
2022-03-18T06:31:21.129 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream"
2022-03-18T06:31:21.129 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="8"
2022-03-18T06:31:21.129 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos"
2022-03-18T06:31:21.129 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora"
2022-03-18T06:31:21.130 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="8"
2022-03-18T06:31:21.130 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el8"
2022-03-18T06:31:21.130 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 8"
2022-03-18T06:31:21.130 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31"
2022-03-18T06:31:21.131 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-03-18T06:31:21.131 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/"
2022-03-18T06:31:21.131 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-18T06:31:21.131 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-18T06:31:21.132 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-03-18T06:31:21.133 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server
2022-03-18T06:31:21.154 INFO:teuthology.lock.ops:Updating [smithi185.front.sepia.ceph.com]: set os type and version on server
2022-03-18T06:31:21.154 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com'
2022-03-18T06:31:21.344 INFO:teuthology.orchestra.run.smithi185.stdout:x86_64
2022-03-18T06:31:21.401 INFO:teuthology.orchestra.run.smithi185.stdout:NAME="CentOS Stream"
2022-03-18T06:31:21.402 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION="8"
2022-03-18T06:31:21.402 INFO:teuthology.orchestra.run.smithi185.stdout:ID="centos"
2022-03-18T06:31:21.402 INFO:teuthology.orchestra.run.smithi185.stdout:ID_LIKE="rhel fedora"
2022-03-18T06:31:21.402 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_ID="8"
2022-03-18T06:31:21.403 INFO:teuthology.orchestra.run.smithi185.stdout:PLATFORM_ID="platform:el8"
2022-03-18T06:31:21.403 INFO:teuthology.orchestra.run.smithi185.stdout:PRETTY_NAME="CentOS Stream 8"
2022-03-18T06:31:21.403 INFO:teuthology.orchestra.run.smithi185.stdout:ANSI_COLOR="0;31"
2022-03-18T06:31:21.403 INFO:teuthology.orchestra.run.smithi185.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-03-18T06:31:21.404 INFO:teuthology.orchestra.run.smithi185.stdout:HOME_URL="https://centos.org/"
2022-03-18T06:31:21.404 INFO:teuthology.orchestra.run.smithi185.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-18T06:31:21.404 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-18T06:31:21.404 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-03-18T06:31:21.405 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server
2022-03-18T06:31:21.506 INFO:teuthology.dispatcher.supervisor:Running job 6743915
2022-03-18T06:31:21.507 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_aa078dbed6d14c1ad98beb5a02b956bbe0a741c8/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743915 --name adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi --description orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743915/orig.config.yaml
2022-03-18T06:31:21.511 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2022-03-18_02:57:32-orch:cephadm-wip-adk-testing-2022-03-17-1930-distro-basic-smithi/6743915
2022-03-18T06:31:21.511 INFO:teuthology.dispatcher.supervisor:Job PID: 20663
2022-03-18T06:31:21.512 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-03-18T13:01:50.290 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-03-18T13:01:50.452 INFO:teuthology.kill:Killing Pids: {20663}
2022-03-18T13:01:51.687 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-03-18T13:01:51.688 INFO:teuthology.task.internal:roles: ubuntu@smithi185.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-03-18T13:01:51.688 INFO:teuthology.misc:Compressing logs...
2022-03-18T13:01:51.689 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com'
2022-03-18T13:01:51.881 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi185.front.sepia.ceph.com'
2022-03-18T13:01:51.910 INFO:teuthology.orchestra.run.smithi139.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-03-18T13:01:52.097 INFO:teuthology.orchestra.run.smithi185.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-03-18T13:01:52.144 INFO:teuthology.orchestra.run.smithi139.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-03-18T13:01:52.144 INFO:teuthology.orchestra.run.smithi139.stderr:tar: Error is not recoverable: exiting now
2022-03-18T13:01:52.196 INFO:teuthology.orchestra.run.smithi185.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-03-18T13:01:52.196 INFO:teuthology.orchestra.run.smithi185.stderr:tar: Error is not recoverable: exiting now
2022-03-18T13:01:52.197 INFO:teuthology.misc:Compressing logs...
2022-03-18T13:01:52.457 INFO:teuthology.misc:Compressing logs...
2022-03-18T13:01:56.461 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/4954d4d6-a686-11ec-8c35-001a4aab830c/ceph-mon.smithi185.log: file size changed while zipping
2022-03-18T13:02:09.170 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/4954d4d6-a686-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-03-18T13:02:13.341 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/4954d4d6-a686-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-03-18T13:02:22.018 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/4954d4d6-a686-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-03-18T13:02:24.487 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/4954d4d6-a686-11ec-8c35-001a4aab830c/ceph-mgr.smithi139.uctdlx.log: file size changed while zipping
2022-03-18T13:02:33.363 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/4954d4d6-a686-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-03-18T13:02:35.541 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/4954d4d6-a686-11ec-8c35-001a4aab830c/ceph-mon.smithi139.log: file size changed while zipping
2022-03-18T13:02:47.304 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/4954d4d6-a686-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-03-18T13:02:54.049 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/4954d4d6-a686-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-03-18T13:03:03.298 INFO:teuthology.orchestra.run.smithi185.stderr:gzip: /var/log/ceph/4954d4d6-a686-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-03-18T13:03:11.161 INFO:teuthology.orchestra.run.smithi139.stderr:gzip: /var/log/ceph/4954d4d6-a686-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-03-18T13:03:27.673 INFO:teuthology.orchestra.run.smithi139.stderr:tar: .: file changed as we read it
2022-03-18T13:03:41.824 INFO:teuthology.kill:No teuthology processes running
2022-03-18T13:03:42.991 INFO:teuthology.kill:Nuking machines: ['smithi139', 'smithi185']
2022-03-18T13:03:44.012 INFO:teuthology.kill:2022-03-18 13:03:44,012.012 INFO:teuthology.nuke:targets:
2022-03-18T13:03:44.013 INFO:teuthology.kill:  smithi139.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDkvkpaTS0HozhSEZ+OjlnEz+aw0Xr485mhjaQZK6e2Kq1s34Uh/ZD5GHUZjlGzupSmV6HKQBxC5HJI1til3dejOwy6So2Grg+BXS79X0vSQy1Q2BHTU3BWvSSSJeZOQU8U8v7J9E97rmjO1My5HFMEgkUKV1rkXqBt+ZaKPSemd19HjzQdcEtF2WcPFL9OBBjbyBUpGyXAFePeZDxcFfL3yDCzj6wdz+BztnupfnoV/NuxRb9M1xXF0v8NkpUCgYz8Q9Kn1ODkK1CMmv4uNuf69qL/jrNi1ODgGDmUms8INbFYhHGymxHKyvLGvfr88MUd/L2lqDHYhmC0CrVpmtqdPom7lMbTn3Y7Vc3zCCj9NAwy05rcg5V2pGOOqjbRWBXLo955RQ3S2/1HTQkWklMlovi2psEjc1dhoCgJNi3Tmx8FetQO7O9AAnHnSykwVeG/VkYCObsnWs3TR3AYkKiKMP/gv00B/V4b0DFYWAx4K8UdV3gqEoh0smvaw8eXnfU=
2022-03-18T13:03:44.014 INFO:teuthology.kill:  smithi185.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfLrwz0+R3xKkNg8Ee2/GAL4KGRa+b+kYWkajIHYX8h1FhMmt6G4uRBJo95S578QDgMnHFRQm2DVLDeI6t/aJNbDcPl4WWJgWga9EeWs7KMfktc+/+HDLcEU1Tzi3WRbGvNg+gjhd6AQ/TxN5rg9BOGIwzWH4zlgl/kpua6jL8RVi+7njjkxMB+rUCLwXkip0M+5UaQ8DMWLo+3gBASVd3yN3DbI1v8jG/FnBaydizPZmghLiK1ivxFMjbxFIPCICKrTAb1vMhB5oPfn7w8+t9OADfoegGmL4jmNMK629tiTb0wFYIYwx5kJbACQoMp2r0KIfNFIX9fYNNziBlQh0S10RmuwO7+N70q2v5edbORyKMGlZRqiWTLnA0xk2ovZch7xWFb0+7eYndnUHBBckyO2OnO6KbzULrQCwYdwopGdNrCeeISJmTu9yWWMxrbQXZr70Hzx3t6Az2nc0V3XDSZJmLXk5G27M8Ih9KRoabyzG/j91TEFX0NOdrB7KoJYM=
2022-03-18T13:03:44.039 INFO:teuthology.kill:2022-03-18 13:03:44,039.039 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-18T13:03:44.043 INFO:teuthology.kill:2022-03-18 13:03:44,043.043 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-18T13:03:44.124 INFO:teuthology.kill:2022-03-18 13:03:44,124.124 INFO:teuthology.orchestra.console:Power off smithi185
2022-03-18T13:03:44.140 INFO:teuthology.kill:2022-03-18 13:03:44,140.140 INFO:teuthology.orchestra.console:Power off smithi139
2022-03-18T13:03:56.616 INFO:teuthology.kill:2022-03-18 13:03:56,615.615 INFO:teuthology.orchestra.console:Power off for smithi185 completed
2022-03-18T13:03:56.622 INFO:teuthology.kill:2022-03-18 13:03:56,622.622 INFO:teuthology.orchestra.console:Power off for smithi139 completed
2022-03-18T13:03:56.764 INFO:teuthology.kill:2022-03-18 13:03:56,763.763 INFO:teuthology.lock.ops:unlocked: smithi185.front.sepia.ceph.com
2022-03-18T13:03:56.767 INFO:teuthology.kill:2022-03-18 13:03:56,767.767 INFO:teuthology.lock.ops:unlocked: smithi139.front.sepia.ceph.com
2022-03-18T13:05:57.071 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-03-18T13:05:57.166 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi139 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-18_07:57:03-fs-wip-vshankar-testing1-20220317-183011-testing-default-smithi/6744668
2022-03-18T13:05:57.166 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi185 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-03-18_07:57:03-fs-wip-vshankar-testing1-20220317-183011-testing-default-smithi/6744668