2021-12-27T14:48:03.081 INFO:root:teuthology version: 1.1.0-95a7d479 2021-12-27T14:48:03.208 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi055.front.sepia.ceph.com' reimaging 2021-12-27T14:48:03.209 INFO:teuthology.lock.ops:Updating [ubuntu@smithi055.front.sepia.ceph.com]: reset os type and version on server 2021-12-27T14:48:03.209 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2021-12-27T14:48:03.228 INFO:teuthology.lock.ops:Node 'ubuntu@smithi055.front.sepia.ceph.com' reimaging is complete 2021-12-27T14:48:03.229 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi137.front.sepia.ceph.com' reimaging 2021-12-27T14:48:03.229 INFO:teuthology.lock.ops:Updating [ubuntu@smithi137.front.sepia.ceph.com]: reset os type and version on server 2021-12-27T14:48:03.230 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2021-12-27T14:48:03.250 INFO:teuthology.lock.ops:Node 'ubuntu@smithi137.front.sepia.ceph.com' reimaging is complete 2021-12-27T14:48:03.443 INFO:teuthology.provision.fog.smithi055:Scheduling deploy of centos 8.stream 2021-12-27T14:48:03.463 INFO:teuthology.provision.fog.smithi137:Scheduling deploy of centos 8.stream 2021-12-27T14:48:03.878 INFO:teuthology.orchestra.console:Power off smithi055 2021-12-27T14:48:03.893 INFO:teuthology.orchestra.console:Power off smithi137 2021-12-27T14:48:12.250 INFO:teuthology.orchestra.console:Power off for smithi055 completed 2021-12-27T14:48:12.351 INFO:teuthology.orchestra.console:Power on smithi055 2021-12-27T14:48:16.369 INFO:teuthology.orchestra.console:Power off for smithi137 completed 2021-12-27T14:48:16.471 INFO:teuthology.orchestra.console:Power on smithi137 2021-12-27T14:48:20.717 INFO:teuthology.orchestra.console:Power on for smithi055 completed 2021-12-27T14:48:20.818 INFO:teuthology.provision.fog.smithi055:Waiting for deploy to finish 2021-12-27T14:48:24.837 INFO:teuthology.orchestra.console:Power on for smithi137 completed 2021-12-27T14:48:24.939 INFO:teuthology.provision.fog.smithi137:Waiting for deploy to finish 2021-12-27T14:52:03.711 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-27T14:52:24.067 INFO:teuthology.provision.fog.smithi055:Node is ready 2021-12-27T14:52:24.089 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055.front.sepia.ceph.com 2021-12-27T14:52:24.147 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55 smithi055.front.sepia.ceph.com smithi055 2021-12-27T14:52:24.511 INFO:teuthology.provision.fog.smithi055:Deploy complete! 2021-12-27T14:52:37.563 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-27T14:52:57.738 INFO:teuthology.provision.fog.smithi137:Node is ready 2021-12-27T14:52:57.759 INFO:teuthology.orchestra.run.smithi137.stdout:smithi126 2021-12-27T14:52:57.817 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.126 smithi126.front.sepia.ceph.com smithi126 2021-12-27T14:52:58.161 INFO:teuthology.provision.fog.smithi137:Deploy complete! 2021-12-27T14:52:58.315 INFO:teuthology.lock.ops:Checking smithi055.front.sepia.ceph.com 2021-12-27T14:52:58.316 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-27T14:52:58.335 INFO:teuthology.lock.ops:Checking smithi137.front.sepia.ceph.com 2021-12-27T14:52:58.336 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-27T14:52:58.355 INFO:teuthology.lock.ops:Updating [smithi055.front.sepia.ceph.com]: set os type and version on server 2021-12-27T14:52:58.356 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2021-12-27T14:52:58.605 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2021-12-27T14:52:58.662 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="CentOS Stream" 2021-12-27T14:52:58.662 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="8" 2021-12-27T14:52:58.663 INFO:teuthology.orchestra.run.smithi055.stdout:ID="centos" 2021-12-27T14:52:58.663 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="rhel fedora" 2021-12-27T14:52:58.663 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="8" 2021-12-27T14:52:58.663 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el8" 2021-12-27T14:52:58.664 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="CentOS Stream 8" 2021-12-27T14:52:58.664 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2021-12-27T14:52:58.664 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-27T14:52:58.664 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://centos.org/" 2021-12-27T14:52:58.664 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-12-27T14:52:58.665 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-12-27T14:52:58.665 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-12-27T14:52:58.666 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2021-12-27T14:52:58.686 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: set os type and version on server 2021-12-27T14:52:58.686 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2021-12-27T14:52:58.874 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2021-12-27T14:52:58.928 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2021-12-27T14:52:58.929 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="8" 2021-12-27T14:52:58.929 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2021-12-27T14:52:58.929 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2021-12-27T14:52:58.930 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="8" 2021-12-27T14:52:58.930 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el8" 2021-12-27T14:52:58.930 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 8" 2021-12-27T14:52:58.930 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2021-12-27T14:52:58.930 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-27T14:52:58.931 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2021-12-27T14:52:58.931 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-12-27T14:52:58.931 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-12-27T14:52:58.931 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-12-27T14:52:58.932 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2021-12-27T14:52:59.031 INFO:teuthology.dispatcher.supervisor:Running job 6585351 2021-12-27T14:52:59.031 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585351 --name teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585351/orig.config.yaml 2021-12-27T14:52:59.035 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-12-26_07:01:03-rados-master-distro-default-smithi/6585351 2021-12-27T14:52:59.038 INFO:teuthology.dispatcher.supervisor:Job PID: 21337 2021-12-27T14:52:59.038 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-12-28T02:53:48.590 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-12-28T02:53:48.859 INFO:teuthology.kill:Killing Pids: {21337} 2021-12-28T02:53:49.950 INFO:teuthology.task.internal:roles: ubuntu@smithi055.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2021-12-28T02:53:49.951 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2021-12-28T02:53:49.951 INFO:teuthology.misc:Compressing logs... 2021-12-28T02:53:49.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2021-12-28T02:53:50.141 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2021-12-28T02:53:50.193 INFO:teuthology.orchestra.run.smithi055.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2021-12-28T02:53:50.377 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2021-12-28T02:53:50.422 INFO:teuthology.orchestra.run.smithi055.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2021-12-28T02:53:50.422 INFO:teuthology.orchestra.run.smithi055.stderr:tar: Error is not recoverable: exiting now 2021-12-28T02:53:50.475 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2021-12-28T02:53:50.476 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now 2021-12-28T02:53:50.477 INFO:teuthology.misc:Compressing logs... 2021-12-28T02:53:51.073 INFO:teuthology.misc:Compressing logs... 2021-12-28T02:53:55.416 INFO:teuthology.orchestra.run.smithi137.stderr:gzip: /var/log/ceph/6a3a5202-6725-11ec-8c31-001a4aab830c/ceph-mgr.smithi137.vtivsc.log: file size changed while zipping 2021-12-28T02:54:05.447 INFO:teuthology.orchestra.run.smithi137.stderr:gzip: /var/log/ceph/6a3a5202-6725-11ec-8c31-001a4aab830c/ceph-mon.smithi137.log: file size changed while zipping 2021-12-28T02:54:19.140 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/6a3a5202-6725-11ec-8c31-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2021-12-28T02:54:34.932 INFO:teuthology.orchestra.run.smithi137.stderr:gzip: /var/log/ceph/6a3a5202-6725-11ec-8c31-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2021-12-28T02:54:44.908 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/6a3a5202-6725-11ec-8c31-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2021-12-28T02:54:59.512 INFO:teuthology.orchestra.run.smithi137.stderr:gzip: /var/log/ceph/6a3a5202-6725-11ec-8c31-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2021-12-28T02:55:02.887 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/6a3a5202-6725-11ec-8c31-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2021-12-28T02:55:24.066 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/6a3a5202-6725-11ec-8c31-001a4aab830c/ceph-mon.smithi055.log: file size changed while zipping 2021-12-28T02:55:24.442 INFO:teuthology.orchestra.run.smithi137.stderr:gzip: /var/log/ceph/6a3a5202-6725-11ec-8c31-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2021-12-28T02:55:45.963 INFO:teuthology.orchestra.run.smithi137.stderr:gzip: /var/log/ceph/6a3a5202-6725-11ec-8c31-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2021-12-28T02:55:51.512 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/6a3a5202-6725-11ec-8c31-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2021-12-28T02:56:39.848 INFO:teuthology.kill:No teuthology processes running 2021-12-28T02:56:40.866 INFO:teuthology.kill:Nuking machines: ['smithi055', 'smithi137'] 2021-12-28T02:56:41.702 INFO:teuthology.kill:2021-12-28 02:56:41,702.702 INFO:teuthology.nuke:targets: 2021-12-28T02:56:41.703 INFO:teuthology.kill: smithi055.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOVq+hSEi9joHQ/GBrzp0Q+ehBBgJIKh4sS8LPiCAPLCbnAS9IGbLbmpTNwzqlRNcBtjW9q2GScq1ResMIUnBTGVUYgmAIoXcohRU6Lu4OHlvgHpkhhdElxcg1kupZYgyywSt4AaR6o67vrKGbYgMv79DisBt5ggQiQ8QoTkrFyJ8BmJ7ciqYoEv5IY3G7Snubnq/EuNgV+lypf3Gs3OfWGQedvNejoV8TY7+MOflQ5KqbWMNAGHSwck7uPCu0qi2IaVxR+S/wF4tW7sssq6tb3JgHEyT7s4oIddFXTTTxJXpkUSNUj/oKBJSfFoDiYsBkHMO1KVZ6Z66LpJBRG+/6aC7/LXdZg9jhi7hpizVvMDkhWtGcJYmiJuJEGbZPx4DgMiUjR5FkmC6qADKrxokk16bHF4M3vgB+sKduPur8vHuJgd/1LuRmMUZzc5DXh1LHI1Z58PjFNj4J5tnm/EYzO7Kn0ZARCz2fo8O899FQdy6dGxZiBRiWbo4sfXnYYQU= 2021-12-28T02:56:41.703 INFO:teuthology.kill: smithi137.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgeSo243rKo5yMFSiwu5a62AmVagN7A0TBaXAeb/+wVQZc7KAmk9uhpyfjocTEDHoxKKyBIF7YwGWm4oHvH3VbKL0vhT9g/7JpGy04DoZzAQRdV4hvXyshZQ6zq/NqOLNiRtZ5343Oy7YKCl++JxkY8s+ORS27+B923AWC3TI7bA/5MP5rGFVIdh6gbGuyz65g1sk2+NGRCaO3fKnhC/ti7++A2GfnqDGx+ucUiChLt2amzRml4lJ9TZAiyYY84imF1FGTyYDRsZzm0gsKm/8cHx/eifALdQFFK0wZrVaOjKxCy3Y8A3K5B/1aFjJ3fGW5oshOmgcrQrS9yVXiZUL8/fJOpFNa3gr945QjFZSQnufiQGzTMFjKXU+ZyjDAZFEo/t//n1yk07wykR/slwo5IqtzVzIyW4ZUp0fnHDrmYixy6nHzGLNjvGoSuJzZH7FA53gIWFUnZ8hBs2AU6/KGhcP0VUGX2ro2QfbZ+i4DI/5lXnzBaVAoIzMMPVm0dvs= 2021-12-28T02:56:41.729 INFO:teuthology.kill:2021-12-28 02:56:41,729.729 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-28T02:56:41.730 INFO:teuthology.kill:2021-12-28 02:56:41,730.730 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-28T02:56:41.804 INFO:teuthology.kill:2021-12-28 02:56:41,804.804 INFO:teuthology.orchestra.console:Power off smithi137 2021-12-28T02:56:41.817 INFO:teuthology.kill:2021-12-28 02:56:41,816.816 INFO:teuthology.orchestra.console:Power off smithi055 2021-12-28T02:56:50.178 INFO:teuthology.kill:2021-12-28 02:56:50,178.178 INFO:teuthology.orchestra.console:Power off for smithi055 completed 2021-12-28T02:56:50.329 INFO:teuthology.kill:2021-12-28 02:56:50,328.328 INFO:teuthology.lock.ops:unlocked: smithi055.front.sepia.ceph.com 2021-12-28T02:56:54.284 INFO:teuthology.kill:2021-12-28 02:56:54,284.284 INFO:teuthology.orchestra.console:Power off for smithi137 completed 2021-12-28T02:56:54.432 INFO:teuthology.kill:2021-12-28 02:56:54,432.432 INFO:teuthology.lock.ops:unlocked: smithi137.front.sepia.ceph.com 2021-12-28T02:58:54.718 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15