2021-12-05T20:51:53.307 INFO:root:teuthology version: 1.1.0-844d1377
2021-12-05T20:51:53.433 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi002.front.sepia.ceph.com' reimaging
2021-12-05T20:51:53.434 INFO:teuthology.lock.ops:Updating [ubuntu@smithi002.front.sepia.ceph.com]: reset os type and version on server
2021-12-05T20:51:53.435 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server
2021-12-05T20:51:53.455 INFO:teuthology.lock.ops:Node 'ubuntu@smithi002.front.sepia.ceph.com' reimaging is complete
2021-12-05T20:51:53.456 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi070.front.sepia.ceph.com' reimaging
2021-12-05T20:51:53.456 INFO:teuthology.lock.ops:Updating [ubuntu@smithi070.front.sepia.ceph.com]: reset os type and version on server
2021-12-05T20:51:53.456 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server
2021-12-05T20:51:53.475 INFO:teuthology.lock.ops:Node 'ubuntu@smithi070.front.sepia.ceph.com' reimaging is complete
2021-12-05T20:51:53.675 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of centos 8.stream
2021-12-05T20:51:53.680 INFO:teuthology.provision.fog.smithi070:Scheduling deploy of centos 8.stream
2021-12-05T20:51:54.015 INFO:teuthology.orchestra.console:Power off smithi070
2021-12-05T20:51:54.055 INFO:teuthology.orchestra.console:Power off smithi002
2021-12-05T20:51:54.165 INFO:teuthology.orchestra.console:Power off for smithi070 completed
2021-12-05T20:51:54.267 INFO:teuthology.orchestra.console:Power on smithi070
2021-12-05T20:51:58.516 INFO:teuthology.orchestra.console:Power on for smithi070 completed
2021-12-05T20:51:58.618 INFO:teuthology.provision.fog.smithi070:Waiting for deploy to finish
2021-12-05T20:52:02.423 INFO:teuthology.orchestra.console:Power off for smithi002 completed
2021-12-05T20:52:02.525 INFO:teuthology.orchestra.console:Power on smithi002
2021-12-05T20:52:10.895 INFO:teuthology.orchestra.console:Power on for smithi002 completed
2021-12-05T20:52:10.996 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish
2021-12-05T20:56:45.637 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-05T20:56:45.912 INFO:teuthology.provision.fog.smithi002:Node is ready
2021-12-05T20:56:45.931 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com
2021-12-05T20:56:45.990 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002
2021-12-05T20:56:46.334 INFO:teuthology.provision.fog.smithi002:Deploy complete!
2021-12-05T20:57:03.367 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-05T20:57:03.667 INFO:teuthology.provision.fog.smithi070:Node is ready
2021-12-05T20:57:03.684 INFO:teuthology.orchestra.run.smithi070.stdout:smithi070.front.sepia.ceph.com
2021-12-05T20:57:03.742 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70 smithi070.front.sepia.ceph.com smithi070
2021-12-05T20:57:04.074 INFO:teuthology.provision.fog.smithi070:Deploy complete!
2021-12-05T20:57:04.229 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com
2021-12-05T20:57:04.230 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-05T20:57:04.249 INFO:teuthology.lock.ops:Checking smithi070.front.sepia.ceph.com
2021-12-05T20:57:04.249 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-05T20:57:04.266 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server
2021-12-05T20:57:04.266 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com'
2021-12-05T20:57:04.513 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64
2021-12-05T20:57:04.570 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream"
2021-12-05T20:57:04.571 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8"
2021-12-05T20:57:04.571 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos"
2021-12-05T20:57:04.571 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora"
2021-12-05T20:57:04.571 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8"
2021-12-05T20:57:04.572 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8"
2021-12-05T20:57:04.572 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 8"
2021-12-05T20:57:04.572 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31"
2021-12-05T20:57:04.572 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-05T20:57:04.573 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/"
2021-12-05T20:57:04.573 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-05T20:57:04.573 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-05T20:57:04.573 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-12-05T20:57:04.575 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server
2021-12-05T20:57:04.593 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: set os type and version on server
2021-12-05T20:57:04.593 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com'
2021-12-05T20:57:04.789 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64
2021-12-05T20:57:04.846 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream"
2021-12-05T20:57:04.846 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="8"
2021-12-05T20:57:04.847 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos"
2021-12-05T20:57:04.847 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora"
2021-12-05T20:57:04.847 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="8"
2021-12-05T20:57:04.847 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el8"
2021-12-05T20:57:04.848 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 8"
2021-12-05T20:57:04.848 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31"
2021-12-05T20:57:04.848 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-05T20:57:04.848 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/"
2021-12-05T20:57:04.848 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-05T20:57:04.849 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-05T20:57:04.849 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-12-05T20:57:04.850 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server
2021-12-05T20:57:04.944 INFO:teuthology.dispatcher.supervisor:Running job 6546902
2021-12-05T20:57:04.945 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi/6546902 --name benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi/6546902/orig.config.yaml
2021-12-05T20:57:04.949 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi/6546902
2021-12-05T20:57:04.950 INFO:teuthology.dispatcher.supervisor:Job PID: 3925
2021-12-05T20:57:04.950 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-12-06T08:57:24.685 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-12-06T08:57:24.866 INFO:teuthology.kill:Killing Pids: {3925}
2021-12-06T08:57:26.105 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2021-12-06T08:57:26.106 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2021-12-06T08:57:26.106 INFO:teuthology.misc:Compressing logs...
2021-12-06T08:57:26.107 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com'
2021-12-06T08:57:26.303 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com'
2021-12-06T08:57:26.357 INFO:teuthology.orchestra.run.smithi002.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2021-12-06T08:57:26.537 INFO:teuthology.orchestra.run.smithi070.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2021-12-06T08:57:26.582 INFO:teuthology.orchestra.run.smithi002.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-12-06T08:57:26.582 INFO:teuthology.orchestra.run.smithi002.stderr:tar: Error is not recoverable: exiting now
2021-12-06T08:57:26.689 INFO:teuthology.orchestra.run.smithi070.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-12-06T08:57:26.690 INFO:teuthology.orchestra.run.smithi070.stderr:tar: Error is not recoverable: exiting now
2021-12-06T08:57:26.691 INFO:teuthology.misc:Compressing logs...
2021-12-06T08:57:27.087 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-12-06T08:57:27.285 INFO:teuthology.misc:Compressing logs...
2021-12-06T08:57:36.279 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/5622d00a-560e-11ec-8c2e-001a4aab830c/ceph-mon.smithi002.log: file size changed while zipping
2021-12-06T08:57:37.152 INFO:teuthology.orchestra.run.smithi070.stderr:gzip: /var/log/ceph/5622d00a-560e-11ec-8c2e-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2021-12-06T08:57:45.216 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/5622d00a-560e-11ec-8c2e-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-12-06T08:57:45.624 INFO:teuthology.orchestra.run.smithi070.stderr:gzip: /var/log/ceph/5622d00a-560e-11ec-8c2e-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-12-06T08:57:53.658 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/5622d00a-560e-11ec-8c2e-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-12-06T08:57:55.331 INFO:teuthology.orchestra.run.smithi070.stderr:gzip: /var/log/ceph/5622d00a-560e-11ec-8c2e-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-12-06T08:57:59.853 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/5622d00a-560e-11ec-8c2e-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-12-06T08:58:03.326 INFO:teuthology.orchestra.run.smithi070.stderr:gzip: /var/log/ceph/5622d00a-560e-11ec-8c2e-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2021-12-06T08:58:07.142 INFO:teuthology.orchestra.run.smithi070.stderr:gzip: /var/log/ceph/5622d00a-560e-11ec-8c2e-001a4aab830c/ceph-mgr.smithi070.yyulqg.log: file size changed while zipping
2021-12-06T08:58:09.184 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/5622d00a-560e-11ec-8c2e-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-12-06T08:58:11.999 INFO:teuthology.orchestra.run.smithi070.stderr:gzip: /var/log/ceph/5622d00a-560e-11ec-8c2e-001a4aab830c/ceph-mon.smithi070.log: file size changed while zipping
2021-12-06T08:59:01.015 INFO:teuthology.kill:No teuthology processes running
2021-12-06T08:59:02.197 INFO:teuthology.kill:Nuking machines: ['smithi002', 'smithi070']
2021-12-06T08:59:03.201 INFO:teuthology.kill:2021-12-06 08:59:03,200.200 INFO:teuthology.nuke:targets:
2021-12-06T08:59:03.202 INFO:teuthology.kill:  smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBa5r/2VjBDc17iGa+Vd4epkW7yr87K29M/UOkHj8wJBie/yYnYiR7dkGkAyB/tH2mK5qeNg5sfCE0omIjf9ujCBb8BXJjdlGVVoVc4tLvH8THBY9/7Q8RCWXt1ys1zex+prKQokodLflsOQHJKzSZMQhtQiLPoktbucBnzza1louyJGVbQgqKv+8cdzcTohNirsD+feR8sUyqOxnjR/hebNvSiyn+gt+mxQcjQ5xrpF722/Rken+KUQQp0tA+9S6+2si9luB5vc+Pj0PRphV+I1XggckvBp3ifYuV2yvzeu1wMkqH20IfuhUUVOLKcF1owwsLrnauXdt0Q1pmIeJFkLIadr+PG47VnVSMTLQKL0StX/hh9ZX+2Ay+fx8yUdh4rJw46TlM//dimbcNsN+b+poyFzd5Sma0ZBViCcwUquU2UYgH5J7gEoT+2Vy+w6Q4K37ag3b4EbMCreEZ/MQtYDumzNvahz8qZAAGghyrJWVSDewalWZWJOtiWJstGZU=
2021-12-06T08:59:03.202 INFO:teuthology.kill:  smithi070.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9zu+YlyB8EDsdYxkxnC1P3SGeYQWBfT3LcXwtoFK+wFrvV+4nj0GxSuCH0bhiR4m/AnATlf/IrkSDHtwc8MMKSeBzUWshX+hk5yzkUQ425m0a+RntBe2gECxKiYa0UsGKJUz+TQdye7vRvdjjN0q8TIVGvkY6zdtk97AZHzR4EzKeBqPzgjQAumt/t+B3AV4cp5lOVP20ihLJrhEgCCsPiSFa7XY2bmqlhA+cBx/Bn18AQKmofwhHQKoU6mafRcBeIEx60MftSbzWqAfGwWg9YGPqg72SLRO9bWHzIBpajGLWt9PJdJjYItbyY3ojoCAW3peoFANn+Pmvm2OLMzmwY5mCZk7WCv1QJUdVXAdsHIYJxJhuS7JUG4a0pNYf71iTTOy/lPoCwriUY4ToeV/Kh3GzZhYQAabeMfmO9C9UV4kqRQnKkB6ANH4JqN9xkVKLLkwYRpCjCnLrA6AImpuCC9Cavc/g6HETAMEpeptuJnJd5TBLljlwJt+fJJdE9c0=
2021-12-06T08:59:03.230 INFO:teuthology.kill:2021-12-06 08:59:03,230.230 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-06T08:59:03.234 INFO:teuthology.kill:2021-12-06 08:59:03,234.234 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-06T08:59:03.309 INFO:teuthology.kill:2021-12-06 08:59:03,309.309 INFO:teuthology.orchestra.console:Power off smithi002
2021-12-06T08:59:03.323 INFO:teuthology.kill:2021-12-06 08:59:03,322.322 INFO:teuthology.orchestra.console:Power off smithi070
2021-12-06T08:59:11.681 INFO:teuthology.kill:2021-12-06 08:59:11,681.681 INFO:teuthology.orchestra.console:Power off for smithi002 completed
2021-12-06T08:59:11.688 INFO:teuthology.kill:2021-12-06 08:59:11,688.688 INFO:teuthology.orchestra.console:Power off for smithi070 completed
2021-12-06T08:59:11.840 INFO:teuthology.kill:2021-12-06 08:59:11,839.839 INFO:teuthology.lock.ops:unlocked: smithi002.front.sepia.ceph.com
2021-12-06T08:59:11.847 INFO:teuthology.kill:2021-12-06 08:59:11,847.847 INFO:teuthology.lock.ops:unlocked: smithi070.front.sepia.ceph.com
2021-12-06T09:01:12.198 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-12-06T09:01:12.303 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi002 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-06_05:05:02-rgw-octopus-distro-default-smithi/6547313
2021-12-06T09:01:12.304 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi070 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-06_05:05:02-rgw-octopus-distro-default-smithi/6547313