2021-11-21T00:27:25.432 INFO:root:teuthology version: 1.1.0-6cfddd5e 2021-11-21T00:27:25.575 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi060.front.sepia.ceph.com' reimaging 2021-11-21T00:27:25.576 INFO:teuthology.lock.ops:Updating [ubuntu@smithi060.front.sepia.ceph.com]: reset os type and version on server 2021-11-21T00:27:25.576 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2021-11-21T00:27:25.596 INFO:teuthology.lock.ops:Node 'ubuntu@smithi060.front.sepia.ceph.com' reimaging is complete 2021-11-21T00:27:25.596 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging 2021-11-21T00:27:25.597 INFO:teuthology.lock.ops:Updating [ubuntu@smithi146.front.sepia.ceph.com]: reset os type and version on server 2021-11-21T00:27:25.597 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2021-11-21T00:27:25.621 INFO:teuthology.lock.ops:Node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging is complete 2021-11-21T00:27:25.823 INFO:teuthology.provision.fog.smithi060:Scheduling deploy of centos 8.stream 2021-11-21T00:27:25.834 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of centos 8.stream 2021-11-21T00:27:26.195 INFO:teuthology.orchestra.console:Power off smithi060 2021-11-21T00:27:26.211 INFO:teuthology.orchestra.console:Power off smithi146 2021-11-21T00:27:34.583 INFO:teuthology.orchestra.console:Power off for smithi060 completed 2021-11-21T00:27:34.685 INFO:teuthology.orchestra.console:Power on smithi060 2021-11-21T00:27:38.701 INFO:teuthology.orchestra.console:Power off for smithi146 completed 2021-11-21T00:27:38.803 INFO:teuthology.orchestra.console:Power on smithi146 2021-11-21T00:27:43.049 INFO:teuthology.orchestra.console:Power on for smithi060 completed 2021-11-21T00:27:43.151 INFO:teuthology.provision.fog.smithi060:Waiting for deploy to finish 2021-11-21T00:27:47.172 INFO:teuthology.orchestra.console:Power on for smithi146 completed 2021-11-21T00:27:47.273 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish 2021-11-21T00:32:17.996 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-21T00:32:18.286 INFO:teuthology.provision.fog.smithi060:Node is ready 2021-11-21T00:32:18.305 INFO:teuthology.orchestra.run.smithi060.stdout:smithi060.front.sepia.ceph.com 2021-11-21T00:32:18.364 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60 smithi060.front.sepia.ceph.com smithi060 2021-11-21T00:32:18.716 INFO:teuthology.provision.fog.smithi060:Deploy complete! 2021-11-21T00:32:52.393 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-21T00:32:52.656 INFO:teuthology.provision.fog.smithi146:Node is ready 2021-11-21T00:32:52.713 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com 2021-11-21T00:32:52.777 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146 2021-11-21T00:32:53.114 INFO:teuthology.provision.fog.smithi146:Deploy complete! 2021-11-21T00:32:53.274 INFO:teuthology.lock.ops:Checking smithi060.front.sepia.ceph.com 2021-11-21T00:32:53.274 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-21T00:32:53.294 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com 2021-11-21T00:32:53.295 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-21T00:32:53.318 INFO:teuthology.lock.ops:Updating [smithi060.front.sepia.ceph.com]: set os type and version on server 2021-11-21T00:32:53.319 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2021-11-21T00:32:53.563 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2021-11-21T00:32:53.620 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream" 2021-11-21T00:32:53.621 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="8" 2021-11-21T00:32:53.621 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos" 2021-11-21T00:32:53.621 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora" 2021-11-21T00:32:53.621 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="8" 2021-11-21T00:32:53.622 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el8" 2021-11-21T00:32:53.622 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 8" 2021-11-21T00:32:53.622 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2021-11-21T00:32:53.622 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-11-21T00:32:53.623 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/" 2021-11-21T00:32:53.623 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-21T00:32:53.623 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-21T00:32:53.626 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-11-21T00:32:53.627 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2021-11-21T00:32:53.650 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server 2021-11-21T00:32:53.650 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2021-11-21T00:32:53.842 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2021-11-21T00:32:53.899 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2021-11-21T00:32:53.899 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8" 2021-11-21T00:32:53.899 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2021-11-21T00:32:53.900 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2021-11-21T00:32:53.900 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8" 2021-11-21T00:32:53.900 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8" 2021-11-21T00:32:53.900 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 8" 2021-11-21T00:32:53.900 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2021-11-21T00:32:53.901 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-11-21T00:32:53.901 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2021-11-21T00:32:53.901 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-21T00:32:53.901 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-21T00:32:53.902 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-11-21T00:32:53.902 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2021-11-21T00:32:54.033 INFO:teuthology.dispatcher.supervisor:Running job 6517005 2021-11-21T00:32:54.033 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-11-20_18:01:41-rados-wip-yuri8-testing-2021-11-20-0807-distro-basic-smithi/6517005 --name yuriw-2021-11-20_18:01:41-rados-wip-yuri8-testing-2021-11-20-0807-distro-basic-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2021-11-20_18:01:41-rados-wip-yuri8-testing-2021-11-20-0807-distro-basic-smithi/6517005/orig.config.yaml 2021-11-21T00:32:54.040 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-11-20_18:01:41-rados-wip-yuri8-testing-2021-11-20-0807-distro-basic-smithi/6517005 2021-11-21T00:32:54.041 INFO:teuthology.dispatcher.supervisor:Job PID: 28427 2021-11-21T00:32:54.041 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-11-21T12:33:47.269 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-11-21T12:33:47.436 INFO:teuthology.kill:Killing Pids: {28427} 2021-11-21T12:33:48.580 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2021-11-21T12:33:48.581 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2021-11-21T12:33:48.581 INFO:teuthology.misc:Compressing logs... 2021-11-21T12:33:48.582 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2021-11-21T12:33:48.788 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2021-11-21T12:33:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2021-11-21T12:33:49.019 INFO:teuthology.orchestra.run.smithi146.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2021-11-21T12:33:49.065 INFO:teuthology.orchestra.run.smithi060.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2021-11-21T12:33:49.066 INFO:teuthology.orchestra.run.smithi060.stderr:tar: Error is not recoverable: exiting now 2021-11-21T12:33:49.163 INFO:teuthology.orchestra.run.smithi146.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2021-11-21T12:33:49.164 INFO:teuthology.orchestra.run.smithi146.stderr:tar: Error is not recoverable: exiting now 2021-11-21T12:33:49.165 INFO:teuthology.misc:Compressing logs... 2021-11-21T12:33:49.392 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-11-21T12:33:49.536 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-11-21T12:33:49.736 INFO:teuthology.misc:Compressing logs... 2021-11-21T12:34:11.638 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/13dc88cc-4a64-11ec-8c2c-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2021-11-21T12:34:15.491 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/13dc88cc-4a64-11ec-8c2c-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2021-11-21T12:34:17.244 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/13dc88cc-4a64-11ec-8c2c-001a4aab830c/ceph-mon.smithi146.log: file size changed while zipping 2021-11-21T12:34:24.945 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/13dc88cc-4a64-11ec-8c2c-001a4aab830c/ceph-mgr.smithi146.ibcsov.log: file size changed while zipping 2021-11-21T12:34:43.008 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/13dc88cc-4a64-11ec-8c2c-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2021-11-21T12:34:49.885 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/13dc88cc-4a64-11ec-8c2c-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2021-11-21T12:35:01.023 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/13dc88cc-4a64-11ec-8c2c-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2021-11-21T12:35:15.384 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/13dc88cc-4a64-11ec-8c2c-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2021-11-21T12:35:28.214 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/13dc88cc-4a64-11ec-8c2c-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2021-11-21T12:35:28.475 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/13dc88cc-4a64-11ec-8c2c-001a4aab830c/ceph-mgr.smithi060.oindob.log: file size changed while zipping 2021-11-21T12:35:45.302 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/13dc88cc-4a64-11ec-8c2c-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2021-11-21T12:35:49.001 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/13dc88cc-4a64-11ec-8c2c-001a4aab830c/ceph-mon.smithi060.log: file size changed while zipping 2021-11-21T12:36:37.340 INFO:teuthology.kill:No teuthology processes running 2021-11-21T12:36:38.397 INFO:teuthology.kill:Nuking machines: ['smithi060', 'smithi146'] 2021-11-21T12:36:39.285 INFO:teuthology.kill:2021-11-21 12:36:39,285.285 INFO:teuthology.nuke:targets: 2021-11-21T12:36:39.286 INFO:teuthology.kill: smithi060.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzjxIQO9FYQxeXGd53cUoWEFpaSAKgWkXjqbkoX6GWHX0wULSCdPJtH+efykj04l4z/nu3uWSqcMHyhHvjlOlbGUkhB8K4OGhTa4wVJUBPZMuNLzr2gtepQ3fsEx84EIFW/AdGrui6oc/idLkBTxkM4oxNuYgsWMc5sfBf3YyyFHv+bAPtD1EP5oUpww/NrAYlRQtQTNqILaubtpOtpMl7EGv1g5mjWyAzT/f1u9mHWwDLo/jK7bi3fucfaQZ8iwweY7UQb8zDliPBUZSamT1zJNoGH44lOTR46ZVfXqOpSKBkjwi1jSo+ty5yclFZ5b+ka+gww6xM1Xh7lBN4G9pTWR63x7A6Y6ZRZoEfPNgXSo7QSxkYGY3RQPPuhoNLyF5wXW9Ma3WxYm5iS179BpkTN2TvD8sI8g211+C7+++qVwYjpEQxxOi/5WXz3lqhCxLFoJckRMdie1SNZKBcu+usXdAYa+/G/IDfvNrhyrk7HizjzCkkzQF3emU14x8MJG0= 2021-11-21T12:36:39.286 INFO:teuthology.kill: smithi146.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3NUlBTOXGrG9ED3nfs8A/EKpZ6+HwPS6zmzV3pDzVZ98VvD78BdklqgrREp7ESoy70LCwI/m8c3UdlSY263K4zbmb/kw7oS7m/Bw3DBgHb3QGjKnDOf0AdTVksRX6F4v82HdNpjop3JNXXPTYgsEVLbvHJqGKeuh7CULpHkoyL2Tn2zf+NR8LAFahNbud/nUQwbBSKYFBYID/xt3fXZ5rBjrQlXcIFK7JwAJy8fNYt587ZHBM6gZw29Bcqonk+PKG4Cdvre1zcsebnZ9GAnTr7bf09wH8jRpN7DlTVLNCPO/Ic9wqfeJ3//IoqPl+YELWhqpxJXQcbZYPVnrS+hC2HJGv/FoPB30vTPSwxHjnlntzvd1eYjS8mey9RRXXrdQvabH0nX2vhqRPObGBGf1A1rnXNMH9JY+/vDJWQYN1tYjpgAyg/6BaggJqk4XkFanIQ+RAX4roBl0InRsvY7PatStNlx2sgVcfPSbNu8CvFavxwGzDKmBpP9u5vlNGuss= 2021-11-21T12:36:39.312 INFO:teuthology.kill:2021-11-21 12:36:39,312.312 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-21T12:36:39.315 INFO:teuthology.kill:2021-11-21 12:36:39,315.315 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-21T12:36:39.389 INFO:teuthology.kill:2021-11-21 12:36:39,388.388 INFO:teuthology.orchestra.console:Power off smithi146 2021-11-21T12:36:39.402 INFO:teuthology.kill:2021-11-21 12:36:39,402.402 INFO:teuthology.orchestra.console:Power off smithi060 2021-11-21T12:36:47.759 INFO:teuthology.kill:2021-11-21 12:36:47,759.759 INFO:teuthology.orchestra.console:Power off for smithi146 completed 2021-11-21T12:36:47.764 INFO:teuthology.kill:2021-11-21 12:36:47,764.764 INFO:teuthology.orchestra.console:Power off for smithi060 completed 2021-11-21T12:36:47.911 INFO:teuthology.kill:2021-11-21 12:36:47,910.910 INFO:teuthology.lock.ops:unlocked: smithi146.front.sepia.ceph.com 2021-11-21T12:36:47.914 INFO:teuthology.kill:2021-11-21 12:36:47,914.914 INFO:teuthology.lock.ops:unlocked: smithi060.front.sepia.ceph.com 2021-11-21T12:38:48.185 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15