2022-02-27T06:44:57.883 INFO:root:teuthology version: 1.1.0-5825e8d0
2022-02-27T06:44:58.011 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi162.front.sepia.ceph.com' reimaging
2022-02-27T06:44:58.012 INFO:teuthology.lock.ops:Updating [ubuntu@smithi162.front.sepia.ceph.com]: reset os type and version on server
2022-02-27T06:44:58.012 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server
2022-02-27T06:44:58.032 INFO:teuthology.lock.ops:Node 'ubuntu@smithi162.front.sepia.ceph.com' reimaging is complete
2022-02-27T06:44:58.032 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi187.front.sepia.ceph.com' reimaging
2022-02-27T06:44:58.032 INFO:teuthology.lock.ops:Updating [ubuntu@smithi187.front.sepia.ceph.com]: reset os type and version on server
2022-02-27T06:44:58.033 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server
2022-02-27T06:44:58.052 INFO:teuthology.lock.ops:Node 'ubuntu@smithi187.front.sepia.ceph.com' reimaging is complete
2022-02-27T06:44:58.245 INFO:teuthology.provision.fog.smithi162:Scheduling deploy of centos 8.stream
2022-02-27T06:44:58.261 INFO:teuthology.provision.fog.smithi187:Scheduling deploy of centos 8.stream
2022-02-27T06:44:58.579 INFO:teuthology.orchestra.console:Power off smithi162
2022-02-27T06:44:58.592 INFO:teuthology.orchestra.console:Power off smithi187
2022-02-27T06:45:11.066 INFO:teuthology.orchestra.console:Power off for smithi162 completed
2022-02-27T06:45:11.074 INFO:teuthology.orchestra.console:Power off for smithi187 completed
2022-02-27T06:45:11.168 INFO:teuthology.orchestra.console:Power on smithi162
2022-02-27T06:45:11.178 INFO:teuthology.orchestra.console:Power on smithi187
2022-02-27T06:45:15.426 INFO:teuthology.orchestra.console:Power on for smithi162 completed
2022-02-27T06:45:15.527 INFO:teuthology.provision.fog.smithi162:Waiting for deploy to finish
2022-02-27T06:45:19.544 INFO:teuthology.orchestra.console:Power on for smithi187 completed
2022-02-27T06:45:19.645 INFO:teuthology.provision.fog.smithi187:Waiting for deploy to finish
2022-02-27T06:49:25.776 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-27T06:49:36.719 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-27T06:49:51.773 INFO:teuthology.provision.fog.smithi187:Node is ready
2022-02-27T06:49:51.796 INFO:teuthology.orchestra.run.smithi187.stdout:smithi187.front.sepia.ceph.com
2022-02-27T06:49:51.856 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.15.187 smithi187.front.sepia.ceph.com smithi187
2022-02-27T06:49:52.230 INFO:teuthology.provision.fog.smithi187:Deploy complete!
2022-02-27T06:50:02.277 INFO:teuthology.provision.fog.smithi162:Node is ready
2022-02-27T06:50:02.301 INFO:teuthology.orchestra.run.smithi162.stdout:smithi162.front.sepia.ceph.com
2022-02-27T06:50:02.361 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162 smithi162.front.sepia.ceph.com smithi162
2022-02-27T06:50:02.731 INFO:teuthology.provision.fog.smithi162:Deploy complete!
2022-02-27T06:50:02.888 INFO:teuthology.lock.ops:Checking smithi162.front.sepia.ceph.com
2022-02-27T06:50:02.889 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-27T06:50:02.911 INFO:teuthology.lock.ops:Checking smithi187.front.sepia.ceph.com
2022-02-27T06:50:02.911 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-27T06:50:02.932 INFO:teuthology.lock.ops:Updating [smithi162.front.sepia.ceph.com]: set os type and version on server
2022-02-27T06:50:02.933 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com'
2022-02-27T06:50:03.124 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64
2022-02-27T06:50:03.181 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream"
2022-02-27T06:50:03.181 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="8"
2022-02-27T06:50:03.182 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos"
2022-02-27T06:50:03.182 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora"
2022-02-27T06:50:03.182 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="8"
2022-02-27T06:50:03.182 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el8"
2022-02-27T06:50:03.183 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-27T06:50:03.183 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31"
2022-02-27T06:50:03.183 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-27T06:50:03.183 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/"
2022-02-27T06:50:03.183 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-27T06:50:03.184 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-27T06:50:03.184 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-27T06:50:03.186 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server
2022-02-27T06:50:03.205 INFO:teuthology.lock.ops:Updating [smithi187.front.sepia.ceph.com]: set os type and version on server
2022-02-27T06:50:03.206 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com'
2022-02-27T06:50:03.447 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64
2022-02-27T06:50:03.504 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="CentOS Stream"
2022-02-27T06:50:03.505 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="8"
2022-02-27T06:50:03.505 INFO:teuthology.orchestra.run.smithi187.stdout:ID="centos"
2022-02-27T06:50:03.505 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="rhel fedora"
2022-02-27T06:50:03.506 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="8"
2022-02-27T06:50:03.506 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el8"
2022-02-27T06:50:03.506 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-27T06:50:03.507 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31"
2022-02-27T06:50:03.507 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-27T06:50:03.507 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://centos.org/"
2022-02-27T06:50:03.508 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-27T06:50:03.508 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-27T06:50:03.508 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-27T06:50:03.509 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server
2022-02-27T06:50:03.631 INFO:teuthology.dispatcher.supervisor:Running job 6708606
2022-02-27T06:50:03.632 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_5825e8d094b6aff4351d26f1ceb5efc598c156a0/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-02-27_03:31:03-rados-pacific-distro-default-smithi/6708606 --name teuthology-2022-02-27_03:31:03-rados-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/teuthology-2022-02-27_03:31:03-rados-pacific-distro-default-smithi/6708606/orig.config.yaml
2022-02-27T06:50:03.635 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-02-27_03:31:03-rados-pacific-distro-default-smithi/6708606
2022-02-27T06:50:03.638 INFO:teuthology.dispatcher.supervisor:Job PID: 30026
2022-02-27T06:50:03.639 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-27T13:20:32.931 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-27T13:20:33.140 INFO:teuthology.kill:Killing Pids: {30026}
2022-02-27T13:20:34.319 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-02-27T13:20:34.320 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-02-27T13:20:34.320 INFO:teuthology.misc:Compressing logs...
2022-02-27T13:20:34.321 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com'
2022-02-27T13:20:34.517 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com'
2022-02-27T13:20:34.568 INFO:teuthology.orchestra.run.smithi162.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-27T13:20:34.755 INFO:teuthology.orchestra.run.smithi187.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-27T13:20:34.804 INFO:teuthology.orchestra.run.smithi162.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-27T13:20:34.805 INFO:teuthology.orchestra.run.smithi162.stderr:tar: Error is not recoverable: exiting now
2022-02-27T13:20:34.845 INFO:teuthology.orchestra.run.smithi187.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-27T13:20:34.846 INFO:teuthology.orchestra.run.smithi187.stderr:tar: Error is not recoverable: exiting now
2022-02-27T13:20:34.847 INFO:teuthology.misc:Compressing logs...
2022-02-27T13:20:35.231 INFO:teuthology.misc:Compressing logs...
2022-02-27T13:20:37.205 INFO:teuthology.orchestra.run.smithi187.stderr:gzip: /var/log/ceph/979805b4-979a-11ec-8c35-001a4aab830c/ceph-mgr.smithi187.naiuxm.log: file size changed while zipping
2022-02-27T13:20:42.027 INFO:teuthology.orchestra.run.smithi187.stderr:gzip: /var/log/ceph/979805b4-979a-11ec-8c35-001a4aab830c/ceph-mon.smithi187.log: file size changed while zipping
2022-02-27T13:20:54.980 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/979805b4-979a-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-02-27T13:20:55.321 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/979805b4-979a-11ec-8c35-001a4aab830c/ceph-mgr.smithi162.lxwptb.log: file size changed while zipping
2022-02-27T13:21:03.274 INFO:teuthology.orchestra.run.smithi187.stderr:gzip: /var/log/ceph/979805b4-979a-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-02-27T13:21:09.244 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/979805b4-979a-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-02-27T13:21:28.603 INFO:teuthology.orchestra.run.smithi187.stderr:gzip: /var/log/ceph/979805b4-979a-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-02-27T13:21:32.456 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/979805b4-979a-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-02-27T13:21:44.204 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/979805b4-979a-11ec-8c35-001a4aab830c/ceph-mon.smithi162.log: file size changed while zipping
2022-02-27T13:21:46.524 INFO:teuthology.orchestra.run.smithi187.stderr:gzip: /var/log/ceph/979805b4-979a-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-02-27T13:22:05.656 INFO:teuthology.orchestra.run.smithi187.stderr:gzip: /var/log/ceph/979805b4-979a-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-02-27T13:22:06.801 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/979805b4-979a-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-02-27T13:22:44.474 INFO:teuthology.kill:No teuthology processes running
2022-02-27T13:22:45.648 INFO:teuthology.kill:Nuking machines: ['smithi162', 'smithi187']
2022-02-27T13:22:46.632 INFO:teuthology.kill:2022-02-27 13:22:46,632.632 INFO:teuthology.nuke:targets:
2022-02-27T13:22:46.633 INFO:teuthology.kill:  smithi162.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCr8DNlLIOfWvtk0fN5Wa8IRFLXfhto7aFyOlfr5fS0SMVAf0itz43JMpPYZ+UE3kJGOGxtXahCwtEpAYmhzgnCHzJ5Qgbwwp+1NC74rkmLjQ65+VcpkuKMSzZbWyLUaYktNuRPfoYf2TnbRqf/3bHNS9KMiaSJns24AAuE2ciAg8SZleuC4BlUYd/mE7LLXcqHFJTqoomAJ/nFmePc5tAfCqDwfuHb0kMCq/jBqy82LVfbXxGi8UlgTXEpj9rOf8ud5jTOvPhGgelMYvLd7RdhAVUC1gopey1svhPl+3M/ibSVfPHbsvkLcIRJUSr2S0muFAEHNdckyDgJJamuEMRnpAB7O2v0VEoOxWwuO+DyYkynPocBiYaKBUNbHx3lZGllMQnZMdtD2aIp/alG7PH9nEhuS7T6SjGG1AkRuCsHqa2bTK7eSreazCv2mete3UIcJCV1mu/vIgLn6nl9o8LxinsriURUMnsp3JKdLhonGMWzXERcK0WdNdkOpLwW84U=
2022-02-27T13:22:46.634 INFO:teuthology.kill:  smithi187.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9+a5yunVHOpeXFCe7hhSDMMLJQM+RYlyZS2wa0fbGSLS6yokFMngeXUyV44fRqjp0ya7I8vFZyahcQ/sVLE+HFx6qQjCQwsPjVQ3p+ZF8Ne7cHHbvRP0xtHmW+6+O5rFvRLMFsxMCYmSIv0jHnXvVyC/OEOsDNhqzUjQkdxWVW+kIcM5JlLBwLP1sFuVPyd+RvY8hqgGaUg+UlADFzj+u9syQ3ZalixaWWn0F9gVUD7bmJ4ynNb2bhkOJfcbuHiQpd5Bh1lm7ARpfw+RSXG2iUWhTJzhBHDqLmT2k5w1+F+Wv3Gg8FGmbZYUh89sJAJLpfsbYCylDanmDlbwYmTN2NdVxUJtreQ8k2aexg4vKhZ9K5rXCBUOlcMXNpydEJ/j/5dvRrzuIs20HbvVyrzY0pxs5E4tGpQPkWq3IRkDXA4lUEL33SJVSpSTCd8wf2XyRwmSShMg6NGp5IAEswIVrgwYVeD8bPdmomXBHinTAnSj4qV7pKHYxgrBSG/sh6ws=
2022-02-27T13:22:46.658 INFO:teuthology.kill:2022-02-27 13:22:46,658.658 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-27T13:22:46.661 INFO:teuthology.kill:2022-02-27 13:22:46,661.661 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-27T13:22:46.742 INFO:teuthology.kill:2022-02-27 13:22:46,742.742 INFO:teuthology.orchestra.console:Power off smithi187
2022-02-27T13:22:46.755 INFO:teuthology.kill:2022-02-27 13:22:46,755.755 INFO:teuthology.orchestra.console:Power off smithi162
2022-02-27T13:22:55.124 INFO:teuthology.kill:2022-02-27 13:22:55,124.124 INFO:teuthology.orchestra.console:Power off for smithi187 completed
2022-02-27T13:22:55.129 INFO:teuthology.kill:2022-02-27 13:22:55,129.129 INFO:teuthology.orchestra.console:Power off for smithi162 completed
2022-02-27T13:22:55.276 INFO:teuthology.kill:2022-02-27 13:22:55,275.275 INFO:teuthology.lock.ops:unlocked: smithi162.front.sepia.ceph.com
2022-02-27T13:22:55.279 INFO:teuthology.kill:2022-02-27 13:22:55,279.279 INFO:teuthology.lock.ops:unlocked: smithi187.front.sepia.ceph.com
2022-02-27T13:24:55.590 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-27T13:24:55.708 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi162 but it was locked by another job: /home/teuthworker/archive/ozeneva-2022-02-27_11:02:05-rados-wip-omri-tracing-compiled-distro-basic-smithi/6709617
2022-02-27T13:24:55.708 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi187 but it was locked by another job: /home/teuthworker/archive/ozeneva-2022-02-27_11:02:05-rados-wip-omri-tracing-compiled-distro-basic-smithi/6709617