2021-12-24T21:42:48.134 INFO:root:teuthology version: 1.1.0-95a7d479
2021-12-24T21:42:48.253 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi164.front.sepia.ceph.com' reimaging
2021-12-24T21:42:48.256 INFO:teuthology.lock.ops:Updating [ubuntu@smithi164.front.sepia.ceph.com]: reset os type and version on server
2021-12-24T21:42:48.256 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server
2021-12-24T21:42:48.278 INFO:teuthology.lock.ops:Node 'ubuntu@smithi164.front.sepia.ceph.com' reimaging is complete
2021-12-24T21:42:48.278 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi169.front.sepia.ceph.com' reimaging
2021-12-24T21:42:48.279 INFO:teuthology.lock.ops:Updating [ubuntu@smithi169.front.sepia.ceph.com]: reset os type and version on server
2021-12-24T21:42:48.279 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server
2021-12-24T21:42:48.298 INFO:teuthology.lock.ops:Node 'ubuntu@smithi169.front.sepia.ceph.com' reimaging is complete
2021-12-24T21:42:48.504 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of centos 8.stream
2021-12-24T21:42:48.507 INFO:teuthology.provision.fog.smithi169:Scheduling deploy of centos 8.stream
2021-12-24T21:42:48.899 INFO:teuthology.orchestra.console:Power off smithi169
2021-12-24T21:42:48.916 INFO:teuthology.orchestra.console:Power off smithi164
2021-12-24T21:42:49.044 INFO:teuthology.orchestra.console:Power off for smithi169 completed
2021-12-24T21:42:49.055 INFO:teuthology.orchestra.console:Power off for smithi164 completed
2021-12-24T21:42:49.146 INFO:teuthology.orchestra.console:Power on smithi169
2021-12-24T21:42:49.156 INFO:teuthology.orchestra.console:Power on smithi164
2021-12-24T21:42:57.509 INFO:teuthology.orchestra.console:Power on for smithi169 completed
2021-12-24T21:42:57.565 INFO:teuthology.orchestra.console:Power on for smithi164 completed
2021-12-24T21:42:57.611 INFO:teuthology.provision.fog.smithi169:Waiting for deploy to finish
2021-12-24T21:42:57.667 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish
2021-12-24T21:47:24.071 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-24T21:47:24.383 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-24T21:47:44.296 INFO:teuthology.provision.fog.smithi164:Node is ready
2021-12-24T21:47:44.317 INFO:teuthology.orchestra.run.smithi164.stdout:smithi126
2021-12-24T21:47:44.375 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.126 smithi126.front.sepia.ceph.com smithi126
2021-12-24T21:47:44.471 INFO:teuthology.provision.fog.smithi169:Node is ready
2021-12-24T21:47:44.492 INFO:teuthology.orchestra.run.smithi169.stdout:smithi169.front.sepia.ceph.com
2021-12-24T21:47:44.549 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.15.169 smithi169.front.sepia.ceph.com smithi169
2021-12-24T21:47:44.746 INFO:teuthology.provision.fog.smithi164:Deploy complete!
2021-12-24T21:47:44.903 INFO:teuthology.provision.fog.smithi169:Deploy complete!
2021-12-24T21:47:45.054 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com
2021-12-24T21:47:45.055 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-24T21:47:45.075 INFO:teuthology.lock.ops:Checking smithi169.front.sepia.ceph.com
2021-12-24T21:47:45.075 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-24T21:47:45.093 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server
2021-12-24T21:47:45.093 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com'
2021-12-24T21:47:45.283 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64
2021-12-24T21:47:45.415 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream"
2021-12-24T21:47:45.416 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8"
2021-12-24T21:47:45.416 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos"
2021-12-24T21:47:45.416 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora"
2021-12-24T21:47:45.417 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8"
2021-12-24T21:47:45.417 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8"
2021-12-24T21:47:45.417 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 8"
2021-12-24T21:47:45.417 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31"
2021-12-24T21:47:45.418 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-24T21:47:45.418 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/"
2021-12-24T21:47:45.418 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-24T21:47:45.418 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-24T21:47:45.419 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-12-24T21:47:45.420 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server
2021-12-24T21:47:45.442 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: set os type and version on server
2021-12-24T21:47:45.442 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com'
2021-12-24T21:47:45.632 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64
2021-12-24T21:47:45.689 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream"
2021-12-24T21:47:45.689 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="8"
2021-12-24T21:47:45.690 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos"
2021-12-24T21:47:45.690 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora"
2021-12-24T21:47:45.690 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="8"
2021-12-24T21:47:45.690 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el8"
2021-12-24T21:47:45.690 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 8"
2021-12-24T21:47:45.691 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31"
2021-12-24T21:47:45.691 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-24T21:47:45.691 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/"
2021-12-24T21:47:45.691 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-24T21:47:45.692 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-24T21:47:45.692 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-12-24T21:47:45.692 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server
2021-12-24T21:47:45.786 INFO:teuthology.dispatcher.supervisor:Running job 6583827
2021-12-24T21:47:45.787 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2021-12-24_15:50:35-rados-master-distro-basic-smithi/6583827 --name sage-2021-12-24_15:50:35-rados-master-distro-basic-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/sage-2021-12-24_15:50:35-rados-master-distro-basic-smithi/6583827/orig.config.yaml
2021-12-24T21:47:45.790 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sage-2021-12-24_15:50:35-rados-master-distro-basic-smithi/6583827
2021-12-24T21:47:45.790 INFO:teuthology.dispatcher.supervisor:Job PID: 13087
2021-12-24T21:47:45.791 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-12-25T09:48:38.515 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-12-25T09:48:38.750 INFO:teuthology.kill:Killing Pids: {13087}
2021-12-25T09:48:39.880 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2021-12-25T09:48:39.881 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2021-12-25T09:48:39.881 INFO:teuthology.misc:Compressing logs...
2021-12-25T09:48:39.882 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com'
2021-12-25T09:48:40.078 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com'
2021-12-25T09:48:40.127 INFO:teuthology.orchestra.run.smithi164.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2021-12-25T09:48:40.323 INFO:teuthology.orchestra.run.smithi169.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2021-12-25T09:48:40.428 INFO:teuthology.orchestra.run.smithi164.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-12-25T09:48:40.428 INFO:teuthology.orchestra.run.smithi164.stderr:tar: Error is not recoverable: exiting now
2021-12-25T09:48:40.471 INFO:teuthology.orchestra.run.smithi169.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-12-25T09:48:40.472 INFO:teuthology.orchestra.run.smithi169.stderr:tar: Error is not recoverable: exiting now
2021-12-25T09:48:40.473 INFO:teuthology.misc:Compressing logs...
2021-12-25T09:48:40.758 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-12-25T09:48:40.830 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-12-25T09:48:41.025 INFO:teuthology.misc:Compressing logs...
2021-12-25T09:48:50.860 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/d90eca2e-6503-11ec-8c31-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-12-25T09:48:51.176 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/d90eca2e-6503-11ec-8c31-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2021-12-25T09:48:59.942 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/d90eca2e-6503-11ec-8c31-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-12-25T09:49:00.738 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/d90eca2e-6503-11ec-8c31-001a4aab830c/ceph-mon.smithi164.log: file size changed while zipping
2021-12-25T09:49:10.066 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/d90eca2e-6503-11ec-8c31-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-12-25T09:49:10.118 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/d90eca2e-6503-11ec-8c31-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-12-25T09:49:12.174 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/d90eca2e-6503-11ec-8c31-001a4aab830c/ceph-mgr.smithi169.wwypzj.log: file size changed while zipping
2021-12-25T09:49:17.092 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/d90eca2e-6503-11ec-8c31-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-12-25T09:49:20.487 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/d90eca2e-6503-11ec-8c31-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2021-12-25T09:49:25.480 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/d90eca2e-6503-11ec-8c31-001a4aab830c/ceph-mon.smithi169.log: file size changed while zipping
2021-12-25T09:49:27.293 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/d90eca2e-6503-11ec-8c31-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-12-25T09:50:15.653 INFO:teuthology.kill:No teuthology processes running
2021-12-25T09:50:16.824 INFO:teuthology.kill:Nuking machines: ['smithi164', 'smithi169']
2021-12-25T09:50:17.798 INFO:teuthology.kill:2021-12-25 09:50:17,798.798 INFO:teuthology.nuke:targets:
2021-12-25T09:50:17.799 INFO:teuthology.kill:  smithi164.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdu/qAsUQYkSjSKcyycehrBcXMzJBchU0iiaMQOLEwMdyIc1JJJ4QuVmculSlIj5kB2TIdzxcPLqmtuXtQ2dj6cho2lJK1OoInbyUKW6Go/rtsHxJ/yS2jBWfjV0bcoIQgZcdM4hhuzmwkFSOz32ZaJpaNp7k5yt42o1YPQpvzeLsPXYcFa9vHEDSeZP4dzdir2Ii24Mu7CETHAiJsq/DMEtZKQ9E5VnSVC42OmTOVfMTU9kTcNNB+gDgn+IJeimGpa2/trOJKmirYdBGjji8/MEBywJKjMZkgESI+siEPxTTaOFffYsYtLYn3KZabiU1ks3z6veWqdJnscWHnOUBlnDW2IKbmOAcyR4fuGu58jwN69xNSFx+sZ2H/9TVX0Yj0qfuv8UzSuq8qBisGwvGdovt4xs1f1FMLVMF/3zg25dpqIhs5532uliX5tDH8hwkJYnT7Og1a80lobjFmdIDvuEpB6y5rDu3sIPbIq4qClg1/fmJz0qmvfZalFh4FugM=
2021-12-25T09:50:17.800 INFO:teuthology.kill:  smithi169.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDV/IbM3cUcLxL8r9AIC0uoMogIWluvPgdYUBGk4wixHxd8tULAW7UypQXR/WfSQbMTt/xtPvIs7EDqEKscEkde1Ferrjicvn/LTAhu3H7HOETbbHQLxpAWBnbid/EpAnrTPOl0Hqh4q0r1aY50ouymgIVQ5CYzWCDKk5ML4vysaVyUkDFuqJxnPzMM7m8utYtK5v62IiuorBySXtxoZVmIGpkPM9TRPisFkv4XTfJbljzVQf7Z0o4BbXD7wZ1kAjU1k2qVRBffDRRggLLr9I6/S3w9iOpsU+Hdp9Wyv9qHa7NeNSrelsGDL6jubCEwbs5+A17PKLw7Y91VNy7WRc9/v9uy7vZBnpuiw+e/+vhiYOYov4hFd2RxxnpHMyt3/WCiqbXbEbaCXsn47ESVA17tnN6EGweUS9qtbmDBdv+kkG432APyvJ5qQqDMlU1pjrRGRm6XYcpzVqmffnHPJAsfcHIq51EAoS+Pq47e906H7WhFrAAeZYjaEqZllIhILvs=
2021-12-25T09:50:17.824 INFO:teuthology.kill:2021-12-25 09:50:17,824.824 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-25T09:50:17.829 INFO:teuthology.kill:2021-12-25 09:50:17,828.828 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-25T09:50:17.905 INFO:teuthology.kill:2021-12-25 09:50:17,905.905 INFO:teuthology.orchestra.console:Power off smithi169
2021-12-25T09:50:17.919 INFO:teuthology.kill:2021-12-25 09:50:17,919.919 INFO:teuthology.orchestra.console:Power off smithi164
2021-12-25T09:50:30.382 INFO:teuthology.kill:2021-12-25 09:50:30,382.382 INFO:teuthology.orchestra.console:Power off for smithi169 completed
2021-12-25T09:50:30.388 INFO:teuthology.kill:2021-12-25 09:50:30,388.388 INFO:teuthology.orchestra.console:Power off for smithi164 completed
2021-12-25T09:50:30.541 INFO:teuthology.kill:2021-12-25 09:50:30,540.540 INFO:teuthology.lock.ops:unlocked: smithi164.front.sepia.ceph.com
2021-12-25T09:50:30.542 INFO:teuthology.kill:2021-12-25 09:50:30,542.542 INFO:teuthology.lock.ops:unlocked: smithi169.front.sepia.ceph.com
2021-12-25T09:52:30.853 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-12-25T09:52:30.951 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi164 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-21_05:15:02-krbd-octopus-testing-default-smithi/6574726
2021-12-25T09:52:30.952 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi169 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-21_05:15:02-krbd-octopus-testing-default-smithi/6574726