2022-01-07T13:14:12.473 INFO:root:teuthology version: 1.1.0-95a7d479 2022-01-07T13:14:12.598 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi072.front.sepia.ceph.com' reimaging 2022-01-07T13:14:12.599 INFO:teuthology.lock.ops:Updating [ubuntu@smithi072.front.sepia.ceph.com]: reset os type and version on server 2022-01-07T13:14:12.599 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2022-01-07T13:14:12.619 INFO:teuthology.lock.ops:Node 'ubuntu@smithi072.front.sepia.ceph.com' reimaging is complete 2022-01-07T13:14:12.620 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi164.front.sepia.ceph.com' reimaging 2022-01-07T13:14:12.620 INFO:teuthology.lock.ops:Updating [ubuntu@smithi164.front.sepia.ceph.com]: reset os type and version on server 2022-01-07T13:14:12.621 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2022-01-07T13:14:12.642 INFO:teuthology.lock.ops:Node 'ubuntu@smithi164.front.sepia.ceph.com' reimaging is complete 2022-01-07T13:14:12.844 INFO:teuthology.provision.fog.smithi072:Scheduling deploy of centos 8.stream 2022-01-07T13:14:12.856 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of centos 8.stream 2022-01-07T13:14:13.254 INFO:teuthology.orchestra.console:Power off smithi164 2022-01-07T13:14:13.271 INFO:teuthology.orchestra.console:Power off smithi072 2022-01-07T13:14:25.740 INFO:teuthology.orchestra.console:Power off for smithi164 completed 2022-01-07T13:14:25.747 INFO:teuthology.orchestra.console:Power off for smithi072 completed 2022-01-07T13:14:25.843 INFO:teuthology.orchestra.console:Power on smithi164 2022-01-07T13:14:25.851 INFO:teuthology.orchestra.console:Power on smithi072 2022-01-07T13:14:30.097 INFO:teuthology.orchestra.console:Power on for smithi164 completed 2022-01-07T13:14:30.102 INFO:teuthology.orchestra.console:Power on for smithi072 completed 2022-01-07T13:14:30.199 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish 2022-01-07T13:14:30.204 INFO:teuthology.provision.fog.smithi072:Waiting for deploy to finish 2022-01-07T13:19:35.229 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-07T13:19:35.507 INFO:teuthology.provision.fog.smithi164:Node is ready 2022-01-07T13:19:35.524 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164.front.sepia.ceph.com 2022-01-07T13:19:35.581 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164 2022-01-07T13:19:35.914 INFO:teuthology.provision.fog.smithi164:Deploy complete! 2022-01-07T13:19:50.594 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-07T13:19:50.969 INFO:teuthology.provision.fog.smithi072:Node is ready 2022-01-07T13:19:50.988 INFO:teuthology.orchestra.run.smithi072.stdout:smithi072.front.sepia.ceph.com 2022-01-07T13:19:51.045 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72 smithi072.front.sepia.ceph.com smithi072 2022-01-07T13:19:51.384 INFO:teuthology.provision.fog.smithi072:Deploy complete! 2022-01-07T13:19:51.532 INFO:teuthology.lock.ops:Checking smithi072.front.sepia.ceph.com 2022-01-07T13:19:51.533 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-07T13:19:51.552 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com 2022-01-07T13:19:51.553 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-07T13:19:51.571 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: set os type and version on server 2022-01-07T13:19:51.571 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2022-01-07T13:19:51.756 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2022-01-07T13:19:51.811 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="CentOS Stream" 2022-01-07T13:19:51.811 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="8" 2022-01-07T13:19:51.811 INFO:teuthology.orchestra.run.smithi072.stdout:ID="centos" 2022-01-07T13:19:51.812 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="rhel fedora" 2022-01-07T13:19:51.812 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="8" 2022-01-07T13:19:51.812 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el8" 2022-01-07T13:19:51.812 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-07T13:19:51.812 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2022-01-07T13:19:51.813 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-07T13:19:51.813 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://centos.org/" 2022-01-07T13:19:51.813 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-07T13:19:51.813 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-07T13:19:51.813 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-07T13:19:51.815 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2022-01-07T13:19:51.833 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server 2022-01-07T13:19:51.834 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2022-01-07T13:19:52.072 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2022-01-07T13:19:52.128 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2022-01-07T13:19:52.129 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8" 2022-01-07T13:19:52.129 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2022-01-07T13:19:52.129 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2022-01-07T13:19:52.129 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8" 2022-01-07T13:19:52.129 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8" 2022-01-07T13:19:52.130 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-07T13:19:52.130 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2022-01-07T13:19:52.130 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-07T13:19:52.130 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2022-01-07T13:19:52.131 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-07T13:19:52.131 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-07T13:19:52.131 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-07T13:19:52.132 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2022-01-07T13:19:52.225 INFO:teuthology.dispatcher.supervisor:Running job 6601029 2022-01-07T13:19:52.226 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_gabrioux@teuthology --archive /home/teuthworker/archive/gabrioux-2022-01-07_12:26:52-rados:cephadm-wip-guits-testing-2022-01-07-0853-pacific-distro-default-smithi/6601029 --name gabrioux-2022-01-07_12:26:52-rados:cephadm-wip-guits-testing-2022-01-07-0853-pacific-distro-default-smithi --description rados:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/gabrioux-2022-01-07_12:26:52-rados:cephadm-wip-guits-testing-2022-01-07-0853-pacific-distro-default-smithi/6601029/orig.config.yaml 2022-01-07T13:19:52.228 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/gabrioux-2022-01-07_12:26:52-rados:cephadm-wip-guits-testing-2022-01-07-0853-pacific-distro-default-smithi/6601029 2022-01-07T13:19:52.229 INFO:teuthology.dispatcher.supervisor:Job PID: 28458 2022-01-07T13:19:52.229 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-08T01:20:45.999 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-01-08T01:20:46.148 INFO:teuthology.kill:Killing Pids: {28458} 2022-01-08T01:20:47.442 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-01-08T01:20:47.443 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-01-08T01:20:47.443 INFO:teuthology.misc:Compressing logs... 2022-01-08T01:20:47.444 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2022-01-08T01:20:47.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2022-01-08T01:20:47.681 INFO:teuthology.orchestra.run.smithi072.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-08T01:20:47.868 INFO:teuthology.orchestra.run.smithi164.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-08T01:20:47.945 INFO:teuthology.orchestra.run.smithi072.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-08T01:20:47.945 INFO:teuthology.orchestra.run.smithi072.stderr:tar: Error is not recoverable: exiting now 2022-01-08T01:20:48.094 INFO:teuthology.orchestra.run.smithi164.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-08T01:20:48.094 INFO:teuthology.orchestra.run.smithi164.stderr:tar: Error is not recoverable: exiting now 2022-01-08T01:20:48.096 INFO:teuthology.misc:Compressing logs... 2022-01-08T01:20:48.359 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-01-08T01:20:48.475 INFO:teuthology.orchestra.run.smithi072.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-01-08T01:20:48.684 INFO:teuthology.misc:Compressing logs... 2022-01-08T01:20:57.625 INFO:teuthology.orchestra.run.smithi072.stderr:gzip: /var/log/ceph/b8c6c90a-6fbd-11ec-8c32-001a4aab830c/ceph-mgr.smithi072.mpqmpz.log: file size changed while zipping 2022-01-08T01:21:19.017 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/b8c6c90a-6fbd-11ec-8c32-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-08T01:21:26.602 INFO:teuthology.orchestra.run.smithi072.stderr:gzip: /var/log/ceph/b8c6c90a-6fbd-11ec-8c32-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-08T01:21:43.977 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/b8c6c90a-6fbd-11ec-8c32-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-08T01:21:48.044 INFO:teuthology.orchestra.run.smithi072.stderr:gzip: /var/log/ceph/b8c6c90a-6fbd-11ec-8c32-001a4aab830c/ceph-mon.smithi072.log: file size changed while zipping 2022-01-08T01:21:50.192 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/b8c6c90a-6fbd-11ec-8c32-001a4aab830c/ceph-mon.smithi164.log: file size changed while zipping 2022-01-08T01:22:11.867 INFO:teuthology.orchestra.run.smithi072.stderr:gzip: /var/log/ceph/b8c6c90a-6fbd-11ec-8c32-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-08T01:22:15.887 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/b8c6c90a-6fbd-11ec-8c32-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-08T01:22:28.748 INFO:teuthology.orchestra.run.smithi072.stderr:gzip: /var/log/ceph/b8c6c90a-6fbd-11ec-8c32-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-08T01:22:37.916 INFO:teuthology.orchestra.run.smithi164.stderr:gzip: /var/log/ceph/b8c6c90a-6fbd-11ec-8c32-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-08T01:22:57.088 INFO:teuthology.orchestra.run.smithi072.stderr:gzip: /var/log/ceph/b8c6c90a-6fbd-11ec-8c32-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-08T01:23:48.061 INFO:teuthology.kill:No teuthology processes running 2022-01-08T01:23:49.227 INFO:teuthology.kill:Nuking machines: ['smithi072', 'smithi164'] 2022-01-08T01:23:50.217 INFO:teuthology.kill:2022-01-08 01:23:50,216.216 INFO:teuthology.nuke:targets: 2022-01-08T01:23:50.218 INFO:teuthology.kill: smithi072.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDH1+w47vt8U+6no/9NZODAt0t7FSDvUrfGsxhE9o6WTmrWoqWnukTUHtq+I57m/eFCcEsNrYkRhDZ6Yt1hhsfb2QLLmtsG3hqbo566svjNBylF5U3BftxdrT0Rf3Pe6kQaJVjeYX8A1hjpC9HrQYeOjiz7OJtkN7AnuCzkHPG6UJyDikNHYr3S2gL0zh6zRZG5YvHDirP7aRZSovxQQw3cXTn5rEpxniUBi98uGqr0dYwagun1ivnzCWB5XLNGsGOfc4ugzkV0o5nppS6GqzNPw6HdAiAGttzD8i8L0W2/cAaBjDRJKx99XOCfc56+6E2vg+xLgpXXCPy5Bbn/grglg8Jp3zRzpytNXlh3O52jg7RWvx6EFtwb90oudq5xi7hxZnvp8qRxh9tCs9qyEVIRnN/XEYQ2HXovON39732/vKewSTWVOYlJlGADtL94v/TKOcp0p39k8hokB1if3ORtAv3Pa4t1jw/vhiwF3sDxMJ8AUietYNpvsnl1dieHy+E= 2022-01-08T01:23:50.218 INFO:teuthology.kill: smithi164.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXsesnwrKcm216kvF94h233r8vEyU6A5vNzOxFTt/rIMYZ2jPXW35GfDIrGSEdhMz+j6HfCDQJKlYZ8E95XiF9MbGKrX1InTYtmEm3RwPlbv8CfdrU4Qz1XgaZQSVUKGlrbXKgRBRM3SRbZGPBIGIlyWswgNnMNXQw5a2+za0uZT/6al/xIZPckfNpUB7cKTh769iYheq5sU1dO62eNlek5g7GTLFOxRVHDca4/jDF1dXoYMXHwHv67pA+KiEQ7DvssOFVf2QUhdfqgW5PSQeb5iI9Onj9lLwFkcLuUSuAja/vDpWBUgdbavWhsEjk2/oCHxfxJFcL4NIAiiEyPF6yH3DbHm6bwOTjlSXQIAwK8peZn0k+nCKaZKxgDKujQCusWPAYWNTwFmPrw0zx56SxNO8deo9GesfvaqipDdcgso2lQIDhXjh/Jz7tOLaZReirANsGFXp9dzRu0u+7Vwm/LUiV/SJ+Cz1AIt6Inf7NksAJu2zCsLruVp1y2GksJFU= 2022-01-08T01:23:50.242 INFO:teuthology.kill:2022-01-08 01:23:50,242.242 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-08T01:23:50.247 INFO:teuthology.kill:2022-01-08 01:23:50,247.247 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-08T01:23:50.326 INFO:teuthology.kill:2022-01-08 01:23:50,326.326 INFO:teuthology.orchestra.console:Power off smithi164 2022-01-08T01:23:50.343 INFO:teuthology.kill:2022-01-08 01:23:50,343.343 INFO:teuthology.orchestra.console:Power off smithi072 2022-01-08T01:23:58.712 INFO:teuthology.kill:2022-01-08 01:23:58,712.712 INFO:teuthology.orchestra.console:Power off for smithi164 completed 2022-01-08T01:23:58.860 INFO:teuthology.kill:2022-01-08 01:23:58,859.859 INFO:teuthology.lock.ops:unlocked: smithi164.front.sepia.ceph.com 2022-01-08T01:24:02.829 INFO:teuthology.kill:2022-01-08 01:24:02,829.829 INFO:teuthology.orchestra.console:Power off for smithi072 completed 2022-01-08T01:24:02.974 INFO:teuthology.kill:2022-01-08 01:24:02,974.974 INFO:teuthology.lock.ops:unlocked: smithi072.front.sepia.ceph.com 2022-01-08T01:26:03.294 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-08T01:26:03.396 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi072 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-07_20:51:42-rados-wip-yuri-testing-2022-01-07-0928-pacific-distro-default-smithi/6601750 2022-01-08T01:26:03.396 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi164 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-07_20:51:42-rados-wip-yuri-testing-2022-01-07-0928-pacific-distro-default-smithi/6601750