2022-01-14T13:21:08.284 INFO:root:teuthology version: 1.1.0-6fc20113 2022-01-14T13:21:08.417 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi002.front.sepia.ceph.com' reimaging 2022-01-14T13:21:08.418 INFO:teuthology.lock.ops:Updating [ubuntu@smithi002.front.sepia.ceph.com]: reset os type and version on server 2022-01-14T13:21:08.418 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2022-01-14T13:21:08.438 INFO:teuthology.lock.ops:Node 'ubuntu@smithi002.front.sepia.ceph.com' reimaging is complete 2022-01-14T13:21:08.439 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi152.front.sepia.ceph.com' reimaging 2022-01-14T13:21:08.439 INFO:teuthology.lock.ops:Updating [ubuntu@smithi152.front.sepia.ceph.com]: reset os type and version on server 2022-01-14T13:21:08.439 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2022-01-14T13:21:08.459 INFO:teuthology.lock.ops:Node 'ubuntu@smithi152.front.sepia.ceph.com' reimaging is complete 2022-01-14T13:21:08.693 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of centos 8.stream 2022-01-14T13:21:08.708 INFO:teuthology.provision.fog.smithi152:Scheduling deploy of centos 8.stream 2022-01-14T13:21:09.118 INFO:teuthology.orchestra.console:Power off smithi152 2022-01-14T13:21:09.135 INFO:teuthology.orchestra.console:Power off smithi002 2022-01-14T13:21:17.498 INFO:teuthology.orchestra.console:Power off for smithi152 completed 2022-01-14T13:21:17.504 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2022-01-14T13:21:17.600 INFO:teuthology.orchestra.console:Power on smithi152 2022-01-14T13:21:17.607 INFO:teuthology.orchestra.console:Power on smithi002 2022-01-14T13:21:25.968 INFO:teuthology.orchestra.console:Power on for smithi152 completed 2022-01-14T13:21:25.973 INFO:teuthology.orchestra.console:Power on for smithi002 completed 2022-01-14T13:21:26.070 INFO:teuthology.provision.fog.smithi152:Waiting for deploy to finish 2022-01-14T13:21:26.074 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish 2022-01-14T13:26:00.724 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-14T13:26:00.974 INFO:teuthology.provision.fog.smithi002:Node is ready 2022-01-14T13:26:00.992 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com 2022-01-14T13:26:01.050 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002 2022-01-14T13:26:01.391 INFO:teuthology.provision.fog.smithi002:Deploy complete! 2022-01-14T13:26:30.808 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-14T13:26:31.112 INFO:teuthology.provision.fog.smithi152:Node is ready 2022-01-14T13:26:31.129 INFO:teuthology.orchestra.run.smithi152.stdout:smithi152.front.sepia.ceph.com 2022-01-14T13:26:31.186 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152 smithi152.front.sepia.ceph.com smithi152 2022-01-14T13:26:31.531 INFO:teuthology.provision.fog.smithi152:Deploy complete! 2022-01-14T13:26:31.687 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com 2022-01-14T13:26:31.688 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-14T13:26:31.712 INFO:teuthology.lock.ops:Checking smithi152.front.sepia.ceph.com 2022-01-14T13:26:31.712 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-14T13:26:31.734 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server 2022-01-14T13:26:31.734 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2022-01-14T13:26:31.996 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2022-01-14T13:26:32.089 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2022-01-14T13:26:32.089 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8" 2022-01-14T13:26:32.089 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2022-01-14T13:26:32.090 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2022-01-14T13:26:32.090 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8" 2022-01-14T13:26:32.090 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8" 2022-01-14T13:26:32.090 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-14T13:26:32.091 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2022-01-14T13:26:32.091 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-14T13:26:32.091 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2022-01-14T13:26:32.091 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-14T13:26:32.092 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-14T13:26:32.092 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-14T13:26:32.093 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2022-01-14T13:26:32.113 INFO:teuthology.lock.ops:Updating [smithi152.front.sepia.ceph.com]: set os type and version on server 2022-01-14T13:26:32.113 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2022-01-14T13:26:32.355 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2022-01-14T13:26:32.412 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="CentOS Stream" 2022-01-14T13:26:32.412 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="8" 2022-01-14T13:26:32.413 INFO:teuthology.orchestra.run.smithi152.stdout:ID="centos" 2022-01-14T13:26:32.413 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="rhel fedora" 2022-01-14T13:26:32.414 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="8" 2022-01-14T13:26:32.414 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el8" 2022-01-14T13:26:32.415 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-14T13:26:32.415 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2022-01-14T13:26:32.415 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-14T13:26:32.416 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://centos.org/" 2022-01-14T13:26:32.416 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-14T13:26:32.417 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-14T13:26:32.417 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-14T13:26:32.418 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2022-01-14T13:26:32.528 INFO:teuthology.dispatcher.supervisor:Running job 6616277 2022-01-14T13:26:32.528 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_6fc2011361437a9dfe4e45b50de224392eed8abc/virtualenv/bin/teuthology -v --owner scheduled_gabrioux@teuthology --archive /home/teuthworker/archive/gabrioux-2022-01-14_12:33:20-rados:cephadm-wip-guits-testing-2022-01-14-001-pacific-distro-default-smithi/6616277 --name gabrioux-2022-01-14_12:33:20-rados:cephadm-wip-guits-testing-2022-01-14-001-pacific-distro-default-smithi --description rados:cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/gabrioux-2022-01-14_12:33:20-rados:cephadm-wip-guits-testing-2022-01-14-001-pacific-distro-default-smithi/6616277/orig.config.yaml 2022-01-14T13:26:32.533 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/gabrioux-2022-01-14_12:33:20-rados:cephadm-wip-guits-testing-2022-01-14-001-pacific-distro-default-smithi/6616277 2022-01-14T13:26:32.534 INFO:teuthology.dispatcher.supervisor:Job PID: 19611 2022-01-14T13:26:32.534 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-14T19:57:02.019 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-01-14T19:57:02.244 INFO:teuthology.kill:Killing Pids: {19611} 2022-01-14T19:57:03.416 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-01-14T19:57:03.417 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-01-14T19:57:03.417 INFO:teuthology.misc:Compressing logs... 2022-01-14T19:57:03.418 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2022-01-14T19:57:03.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2022-01-14T19:57:03.641 INFO:teuthology.orchestra.run.smithi002.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-14T19:57:03.850 INFO:teuthology.orchestra.run.smithi152.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-14T19:57:03.899 INFO:teuthology.orchestra.run.smithi002.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-14T19:57:03.900 INFO:teuthology.orchestra.run.smithi002.stderr:tar: Error is not recoverable: exiting now 2022-01-14T19:57:03.981 INFO:teuthology.orchestra.run.smithi152.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-14T19:57:03.982 INFO:teuthology.orchestra.run.smithi152.stderr:tar: Error is not recoverable: exiting now 2022-01-14T19:57:03.983 INFO:teuthology.misc:Compressing logs... 2022-01-14T19:57:04.397 INFO:teuthology.misc:Compressing logs... 2022-01-14T19:57:17.109 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/c7663726-753e-11ec-8c32-001a4aab830c/ceph-mon.smithi002.log: file size changed while zipping 2022-01-14T19:57:24.225 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/c7663726-753e-11ec-8c32-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-14T19:57:37.166 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/c7663726-753e-11ec-8c32-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-14T19:57:40.915 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/c7663726-753e-11ec-8c32-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-14T19:57:42.398 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/c7663726-753e-11ec-8c32-001a4aab830c/ceph-mgr.smithi002.dukpdy.log: file size changed while zipping 2022-01-14T19:57:44.271 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/c7663726-753e-11ec-8c32-001a4aab830c/ceph-mon.smithi152.log: file size changed while zipping 2022-01-14T19:57:59.069 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/c7663726-753e-11ec-8c32-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-14T19:58:00.188 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/c7663726-753e-11ec-8c32-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-14T19:58:10.836 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/c7663726-753e-11ec-8c32-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-14T19:58:14.381 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/c7663726-753e-11ec-8c32-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-14T19:58:29.993 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/c7663726-753e-11ec-8c32-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-14T19:59:01.565 INFO:teuthology.kill:No teuthology processes running 2022-01-14T19:59:02.679 INFO:teuthology.kill:Nuking machines: ['smithi002', 'smithi152'] 2022-01-14T19:59:03.624 INFO:teuthology.kill:2022-01-14 19:59:03,623.623 INFO:teuthology.nuke:targets: 2022-01-14T19:59:03.624 INFO:teuthology.kill: smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCREAvJi6kuHaSIEA7oJ5nwKS+h2sYZbXURQnBlSf8mHXMMbrzw1THdwmIADAtu6OvqZa0KU167xCeqNYFM3N4uXgK2j8qrWhHai6gKzBvYWD7nz9VbiNaW8VZl8B/3761T1wy08AxMmOo/kEElhOfzKID9y7QXj9tOu8F7pqYmDLBc68qnV/6UXrapkEFKdJ8dMXY5b71MleNwN796t3JwBIvA1FK5+gHXg/zmxs2eiKCloOnCPOQQ6OFgEKIMDJbaf6JhRCnEYgMEuioSugR2TLH8RF3IkNqSBjF1PAX8qjFmOLyX3XIegeynN8F2xTPD3gN4LqOdqNVBjhnoIpP9kyrvFwbcDFzPqNoV5e548K1JOkd4pX2fKHl8ImEW8kG+FHS9oyuO0Sop4QDbFNRUpDAqSCmu9RrVLd/4XMwF1AVBaC6wm4YUzTcJK3vIQeEfup/SexQPR5jsxU90GzbMiFZZIC+xWhwG4p3mmhF7KPBjCeGQ33zUU7d4GovoQeU= 2022-01-14T19:59:03.625 INFO:teuthology.kill: smithi152.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqqCc31r0wYIs/U/zT2cSpQpanQpKOSbX8pjUNbJy92EnADoLYpOtFRBbixftVjTeALQyOvUJxFibQjkY85LXcIbGL4zlA6D0ehaPkEiyHLAiUXjpVXu1Y5O1VZu6qUJXHhFYj0r9zXPbnFAzP34JvyCAvEJc1pocCJUX/wBYaEwE3cOhhc+CH3o9C+Qops0j5fPUK7Tq3MWjlqozRhMV7mK3xCPigSbqOovr34LwE0okXmcST/i+IFPRANPTIuVF6B8sBUK3wZa0/NvS3WWiCxqQ5TQ4l9A6N0dcyn9pOMXHD2mPPxetHKUuI0Hlxj5CSF4A1kylr88o1WQf/eGxE5KPxa9tB+cfr/BKzdBfeP6CKvCOeKdL8vCfSPe3qWy4TZkgBA/6Xf91Sw8l025EmIpQ3mWnpc7sehikN8oZQsYGx+24EoeDscrQxmojALtpAm3l1piN7GpwOPBpzbadH2yC6LPgYOk1nkm/1CFcsoJVR06pwbtTUdQBlUvEaRSs= 2022-01-14T19:59:03.650 INFO:teuthology.kill:2022-01-14 19:59:03,649.649 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-14T19:59:03.653 INFO:teuthology.kill:2022-01-14 19:59:03,652.652 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-14T19:59:03.728 INFO:teuthology.kill:2022-01-14 19:59:03,727.727 INFO:teuthology.orchestra.console:Power off smithi002 2022-01-14T19:59:03.742 INFO:teuthology.kill:2022-01-14 19:59:03,742.742 INFO:teuthology.orchestra.console:Power off smithi152 2022-01-14T19:59:12.096 INFO:teuthology.kill:2022-01-14 19:59:12,095.095 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2022-01-14T19:59:12.243 INFO:teuthology.kill:2022-01-14 19:59:12,242.242 INFO:teuthology.lock.ops:unlocked: smithi002.front.sepia.ceph.com 2022-01-14T19:59:16.214 INFO:teuthology.kill:2022-01-14 19:59:16,213.213 INFO:teuthology.orchestra.console:Power off for smithi152 completed 2022-01-14T19:59:16.359 INFO:teuthology.kill:2022-01-14 19:59:16,359.359 INFO:teuthology.lock.ops:unlocked: smithi152.front.sepia.ceph.com 2022-01-14T20:01:16.660 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-14T20:01:16.763 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi002 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-14_18:48:30-rados-wip-yuri6-testing-2022-01-14-0909-distro-default-smithi/6616592 2022-01-14T20:01:16.764 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi152 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-14_18:48:30-rados-wip-yuri6-testing-2022-01-14-0909-distro-default-smithi/6616590