2021-10-28T23:29:55.739 INFO:root:teuthology version: 1.1.0-c56135d1
2021-10-28T23:29:55.867 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi085.front.sepia.ceph.com' reimaging
2021-10-28T23:29:55.868 INFO:teuthology.lock.ops:Updating [ubuntu@smithi085.front.sepia.ceph.com]: reset os type and version on server
2021-10-28T23:29:55.868 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server
2021-10-28T23:29:55.888 INFO:teuthology.lock.ops:Node 'ubuntu@smithi085.front.sepia.ceph.com' reimaging is complete
2021-10-28T23:29:55.889 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi167.front.sepia.ceph.com' reimaging
2021-10-28T23:29:55.889 INFO:teuthology.lock.ops:Updating [ubuntu@smithi167.front.sepia.ceph.com]: reset os type and version on server
2021-10-28T23:29:55.889 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server
2021-10-28T23:29:55.909 INFO:teuthology.lock.ops:Node 'ubuntu@smithi167.front.sepia.ceph.com' reimaging is complete
2021-10-28T23:29:56.091 INFO:teuthology.provision.fog.smithi085:Scheduling deploy of centos 8.2
2021-10-28T23:29:56.110 INFO:teuthology.provision.fog.smithi167:Scheduling deploy of centos 8.2
2021-10-28T23:29:56.470 INFO:teuthology.orchestra.console:Power off smithi167
2021-10-28T23:29:56.509 INFO:teuthology.orchestra.console:Power off smithi085
2021-10-28T23:30:04.845 INFO:teuthology.orchestra.console:Power off for smithi167 completed
2021-10-28T23:30:04.946 INFO:teuthology.orchestra.console:Power on smithi167
2021-10-28T23:30:08.994 INFO:teuthology.orchestra.console:Power off for smithi085 completed
2021-10-28T23:30:09.096 INFO:teuthology.orchestra.console:Power on smithi085
2021-10-28T23:30:13.348 INFO:teuthology.orchestra.console:Power on for smithi085 completed
2021-10-28T23:30:13.349 INFO:teuthology.orchestra.console:Power on for smithi167 completed
2021-10-28T23:30:13.450 INFO:teuthology.provision.fog.smithi085:Waiting for deploy to finish
2021-10-28T23:30:13.452 INFO:teuthology.provision.fog.smithi167:Waiting for deploy to finish
2021-10-28T23:34:25.573 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-28T23:34:45.915 INFO:teuthology.provision.fog.smithi167:Node is ready
2021-10-28T23:34:45.935 INFO:teuthology.orchestra.run.smithi167.stdout:smithi167.front.sepia.ceph.com
2021-10-28T23:34:45.992 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.167 smithi167.front.sepia.ceph.com smithi167
2021-10-28T23:34:46.352 INFO:teuthology.provision.fog.smithi167:Deploy complete!
2021-10-28T23:35:18.072 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-28T23:35:18.346 INFO:teuthology.provision.fog.smithi085:Node is ready
2021-10-28T23:35:18.402 INFO:teuthology.orchestra.run.smithi085.stdout:smithi085.front.sepia.ceph.com
2021-10-28T23:35:18.457 INFO:teuthology.orchestra.run.smithi085.stdout:172.21.15.85 smithi085.front.sepia.ceph.com smithi085
2021-10-28T23:35:18.781 INFO:teuthology.provision.fog.smithi085:Deploy complete!
2021-10-28T23:35:18.955 INFO:teuthology.lock.ops:Checking smithi085.front.sepia.ceph.com
2021-10-28T23:35:18.956 INFO:teuthology.lock.ops:New key found. Updating...
2021-10-28T23:35:18.979 INFO:teuthology.lock.ops:Checking smithi167.front.sepia.ceph.com
2021-10-28T23:35:18.979 INFO:teuthology.lock.ops:New key found. Updating...
2021-10-28T23:35:19.001 INFO:teuthology.lock.ops:Updating [smithi085.front.sepia.ceph.com]: set os type and version on server
2021-10-28T23:35:19.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi085.front.sepia.ceph.com'
2021-10-28T23:35:19.314 INFO:teuthology.orchestra.run.smithi085.stdout:x86_64
2021-10-28T23:35:19.370 INFO:teuthology.orchestra.run.smithi085.stdout:NAME="CentOS Linux"
2021-10-28T23:35:19.370 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION="8 (Core)"
2021-10-28T23:35:19.371 INFO:teuthology.orchestra.run.smithi085.stdout:ID="centos"
2021-10-28T23:35:19.371 INFO:teuthology.orchestra.run.smithi085.stdout:ID_LIKE="rhel fedora"
2021-10-28T23:35:19.371 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION_ID="8"
2021-10-28T23:35:19.372 INFO:teuthology.orchestra.run.smithi085.stdout:PLATFORM_ID="platform:el8"
2021-10-28T23:35:19.372 INFO:teuthology.orchestra.run.smithi085.stdout:PRETTY_NAME="CentOS Linux 8 (Core)"
2021-10-28T23:35:19.372 INFO:teuthology.orchestra.run.smithi085.stdout:ANSI_COLOR="0;31"
2021-10-28T23:35:19.372 INFO:teuthology.orchestra.run.smithi085.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-10-28T23:35:19.372 INFO:teuthology.orchestra.run.smithi085.stdout:HOME_URL="https://www.centos.org/"
2021-10-28T23:35:19.373 INFO:teuthology.orchestra.run.smithi085.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-10-28T23:35:19.373 INFO:teuthology.orchestra.run.smithi085.stdout:
2021-10-28T23:35:19.373 INFO:teuthology.orchestra.run.smithi085.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-10-28T23:35:19.374 INFO:teuthology.orchestra.run.smithi085.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-10-28T23:35:19.374 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT="centos"
2021-10-28T23:35:19.374 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8"
2021-10-28T23:35:19.374 INFO:teuthology.orchestra.run.smithi085.stdout:
2021-10-28T23:35:19.376 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server
2021-10-28T23:35:19.397 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: set os type and version on server
2021-10-28T23:35:19.399 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com'
2021-10-28T23:35:19.664 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64
2021-10-28T23:35:19.719 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Linux"
2021-10-28T23:35:19.719 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8 (Core)"
2021-10-28T23:35:19.720 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos"
2021-10-28T23:35:19.720 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora"
2021-10-28T23:35:19.720 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8"
2021-10-28T23:35:19.720 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8"
2021-10-28T23:35:19.720 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Linux 8 (Core)"
2021-10-28T23:35:19.721 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31"
2021-10-28T23:35:19.721 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-10-28T23:35:19.721 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.centos.org/"
2021-10-28T23:35:19.721 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-10-28T23:35:19.722 INFO:teuthology.orchestra.run.smithi167.stdout:
2021-10-28T23:35:19.722 INFO:teuthology.orchestra.run.smithi167.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-10-28T23:35:19.722 INFO:teuthology.orchestra.run.smithi167.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-10-28T23:35:19.722 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="centos"
2021-10-28T23:35:19.722 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8"
2021-10-28T23:35:19.723 INFO:teuthology.orchestra.run.smithi167.stdout:
2021-10-28T23:35:19.723 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server
2021-10-28T23:35:19.839 INFO:teuthology.dispatcher.supervisor:Running job 6465525
2021-10-28T23:35:19.840 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/bin/teuthology -v --owner scheduled_nojha@teuthology --archive /home/teuthworker/archive/nojha-2021-10-28_21:32:34-rados-wip-43687-distro-basic-smithi/6465525 --name nojha-2021-10-28_21:32:34-rados-wip-43687-distro-basic-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/nojha-2021-10-28_21:32:34-rados-wip-43687-distro-basic-smithi/6465525/orig.config.yaml
2021-10-28T23:35:19.843 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nojha-2021-10-28_21:32:34-rados-wip-43687-distro-basic-smithi/6465525
2021-10-28T23:35:19.844 INFO:teuthology.dispatcher.supervisor:Job PID: 8808
2021-10-28T23:35:19.844 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-10-29T11:36:13.683 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-10-29T11:36:13.868 INFO:teuthology.kill:Killing Pids: {8808}
2021-10-29T11:36:15.013 INFO:teuthology.task.internal:roles: ubuntu@smithi085.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2021-10-29T11:36:15.014 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2021-10-29T11:36:15.014 INFO:teuthology.misc:Compressing logs...
2021-10-29T11:36:15.015 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi085.front.sepia.ceph.com'
2021-10-29T11:36:15.215 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com'
2021-10-29T11:36:15.244 INFO:teuthology.orchestra.run.smithi085.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2021-10-29T11:36:15.507 INFO:teuthology.orchestra.run.smithi167.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2021-10-29T11:36:15.547 INFO:teuthology.orchestra.run.smithi085.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-10-29T11:36:15.548 INFO:teuthology.orchestra.run.smithi085.stderr:tar: Error is not recoverable: exiting now
2021-10-29T11:36:15.580 INFO:teuthology.orchestra.run.smithi167.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-10-29T11:36:15.580 INFO:teuthology.orchestra.run.smithi167.stderr:tar: Error is not recoverable: exiting now
2021-10-29T11:36:15.582 INFO:teuthology.misc:Compressing logs...
2021-10-29T11:36:16.112 INFO:teuthology.orchestra.run.smithi085.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-10-29T11:36:16.216 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-10-29T11:36:16.557 INFO:teuthology.misc:Compressing logs...
2021-10-29T11:36:43.855 INFO:teuthology.orchestra.run.smithi085.stderr:gzip: /var/log/ceph/1fb1768a-3849-11ec-8c28-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-10-29T11:36:45.327 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/1fb1768a-3849-11ec-8c28-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-10-29T11:36:52.189 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/1fb1768a-3849-11ec-8c28-001a4aab830c/ceph-mon.smithi167.log: file size changed while zipping
2021-10-29T11:37:18.048 INFO:teuthology.orchestra.run.smithi085.stderr:gzip: /var/log/ceph/1fb1768a-3849-11ec-8c28-001a4aab830c/ceph-mon.smithi085.log: file size changed while zipping
2021-10-29T11:37:20.238 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/1fb1768a-3849-11ec-8c28-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-10-29T11:37:44.431 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/1fb1768a-3849-11ec-8c28-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2021-10-29T11:37:47.460 INFO:teuthology.orchestra.run.smithi085.stderr:gzip: /var/log/ceph/1fb1768a-3849-11ec-8c28-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-10-29T11:37:57.490 INFO:teuthology.orchestra.run.smithi085.stderr:gzip: /var/log/ceph/1fb1768a-3849-11ec-8c28-001a4aab830c/ceph-mgr.smithi085.zqfhtg.log: file size changed while zipping
2021-10-29T11:38:17.657 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/1fb1768a-3849-11ec-8c28-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2021-10-29T11:38:26.323 INFO:teuthology.orchestra.run.smithi085.stderr:gzip: /var/log/ceph/1fb1768a-3849-11ec-8c28-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-10-29T11:38:55.059 INFO:teuthology.orchestra.run.smithi085.stderr:tar: .: file changed as we read it
2021-10-29T11:39:20.105 INFO:teuthology.kill:No teuthology processes running
2021-10-29T11:39:21.408 INFO:teuthology.kill:Nuking machines: ['smithi085', 'smithi167']
2021-10-29T11:39:22.406 INFO:teuthology.kill:2021-10-29 11:39:22,405.405 INFO:teuthology.nuke:targets:
2021-10-29T11:39:22.407 INFO:teuthology.kill:  smithi085.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwv1eea+HSbpNI5nCB67xx4sQw4JjdHAECMF8cXw/ecQOhbylFMCarkYPnD+jkOrByhoNTdbpFmVsbxvDx88676MXPtOeiqA29wzMErvtrGnD12S4sQ0ja9+KBYx1vL1L+K0bh7Iw+x0Hp2IPxVzhQzBEFRKE1DtD+iGMDCTSxXZb+kxOWxFhK9dc2HVqW6qlF2i1BU7y5tpo2rL8F/5b4GSTAcEGKy7YpxcPlSe5JtdYsGw9da0LCeFUGhZzEgBw1NPegtXmTOoep+OXv19rvsptEMnmu3MxuUyjHJob4fZqhwRvHnGEIW7DMD7OlNIHNWtz6VeNSfjd0UjCLm9EK1DBQXZrtOA9+J1lZckUf/jQa2cLGl6SSeX6gEnjWHivyo1gn21Pec/LR+ccOjLv7zCo6bZy5K8zgevoqA5b72HcS8iVLj/PWU3SMTI2oXgo+z28qT+Nbb6qq6gAOmZzCh73E/dzAWsa6Wfoe5BGz7YxZ7nnAzzzctQ8cU4GQBkk=
2021-10-29T11:39:22.407 INFO:teuthology.kill:  smithi167.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ8lPeqp0o4gWhFIxtET7m2TVLTYss6g8VPcTTOaMDWvdFNeRh+1DCRuAnl8lj0sYarBJrg3hS5EVcm4Eb7UsY0nFFqeCMr1To6xp3EpAdvOxdLb4bnaAuEqH/7EmoJVFDAyfXI5LcDckWYhjuIWXowBemhNKkXmpVD1IGENsC0c0PMwfMU2BrIejkqoUbEDvQuy5flg8m6CHdyX10Rf/Rb5pH0HYwmPqG97PSQvjU/soMkxBAWfwgloVP9NLcGc7V8eIr04chkw3kpJIBaf6GNiEQgUUS8k1ZCj2egq2y9vdp4XSJkiGIQxJ6g/WEoEi1OhDNCB+8akgrPK56WmgP4R0aI9Xci3qFDz6vctJgEtLaWpwiFNyH7yDn4PM5KzOwNJ9ICEE0WCeW21chGaLt5m/GAF+mM+0g/t2eU0pJcZp/DdxmrBQvuzDbMjltv0LY9UTu0695ebkk1Dqy8YNu9oOv4VJF6NjFqt7jpYIFtYeh3lZ+0UJ9Y+pPLQGQoj8=
2021-10-29T11:39:22.434 INFO:teuthology.kill:2021-10-29 11:39:22,434.434 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-29T11:39:22.435 INFO:teuthology.kill:2021-10-29 11:39:22,435.435 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-29T11:39:22.507 INFO:teuthology.kill:2021-10-29 11:39:22,507.507 INFO:teuthology.orchestra.console:Power off smithi085
2021-10-29T11:39:22.520 INFO:teuthology.kill:2021-10-29 11:39:22,520.520 INFO:teuthology.orchestra.console:Power off smithi167
2021-10-29T11:39:34.992 INFO:teuthology.kill:2021-10-29 11:39:34,992.992 INFO:teuthology.orchestra.console:Power off for smithi085 completed
2021-10-29T11:39:34.996 INFO:teuthology.kill:2021-10-29 11:39:34,996.996 INFO:teuthology.orchestra.console:Power off for smithi167 completed
2021-10-29T11:39:35.143 INFO:teuthology.kill:2021-10-29 11:39:35,143.143 INFO:teuthology.lock.ops:unlocked: smithi085.front.sepia.ceph.com
2021-10-29T11:39:35.147 INFO:teuthology.kill:2021-10-29 11:39:35,147.147 INFO:teuthology.lock.ops:unlocked: smithi167.front.sepia.ceph.com
2021-10-29T11:41:35.469 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-10-29T11:41:35.568 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi085 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-10-28_23:09:03-rados-wip-yuri2-testing-2021-10-28-1343-distro-basic-smithi/6466171
2021-10-29T11:41:35.568 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi167 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-10-28_23:09:03-rados-wip-yuri2-testing-2021-10-28-1343-distro-basic-smithi/6466171