2022-01-06T10:51:16.669 INFO:root:teuthology version: 1.1.0-95a7d479 2022-01-06T10:51:16.804 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi055.front.sepia.ceph.com' reimaging 2022-01-06T10:51:16.805 INFO:teuthology.lock.ops:Updating [ubuntu@smithi055.front.sepia.ceph.com]: reset os type and version on server 2022-01-06T10:51:16.805 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2022-01-06T10:51:16.825 INFO:teuthology.lock.ops:Node 'ubuntu@smithi055.front.sepia.ceph.com' reimaging is complete 2022-01-06T10:51:16.825 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi152.front.sepia.ceph.com' reimaging 2022-01-06T10:51:16.826 INFO:teuthology.lock.ops:Updating [ubuntu@smithi152.front.sepia.ceph.com]: reset os type and version on server 2022-01-06T10:51:16.826 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2022-01-06T10:51:16.845 INFO:teuthology.lock.ops:Node 'ubuntu@smithi152.front.sepia.ceph.com' reimaging is complete 2022-01-06T10:51:17.027 INFO:teuthology.provision.fog.smithi055:Scheduling deploy of centos 8.stream 2022-01-06T10:51:17.039 INFO:teuthology.provision.fog.smithi152:Scheduling deploy of centos 8.stream 2022-01-06T10:51:17.356 INFO:teuthology.orchestra.console:Power off smithi055 2022-01-06T10:51:17.370 INFO:teuthology.orchestra.console:Power off smithi152 2022-01-06T10:51:29.840 INFO:teuthology.orchestra.console:Power off for smithi055 completed 2022-01-06T10:51:29.845 INFO:teuthology.orchestra.console:Power off for smithi152 completed 2022-01-06T10:51:29.942 INFO:teuthology.orchestra.console:Power on smithi055 2022-01-06T10:51:29.950 INFO:teuthology.orchestra.console:Power on smithi152 2022-01-06T10:51:34.196 INFO:teuthology.orchestra.console:Power on for smithi055 completed 2022-01-06T10:51:34.297 INFO:teuthology.provision.fog.smithi055:Waiting for deploy to finish 2022-01-06T10:51:38.313 INFO:teuthology.orchestra.console:Power on for smithi152 completed 2022-01-06T10:51:38.414 INFO:teuthology.provision.fog.smithi152:Waiting for deploy to finish 2022-01-06T10:56:05.645 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-06T10:56:09.039 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-01-06T10:56:09.304 INFO:teuthology.provision.fog.smithi055:Node is ready 2022-01-06T10:56:09.361 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055.front.sepia.ceph.com 2022-01-06T10:56:09.419 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55 smithi055.front.sepia.ceph.com smithi055 2022-01-06T10:56:09.756 INFO:teuthology.provision.fog.smithi055:Deploy complete! 2022-01-06T10:56:30.581 INFO:teuthology.provision.fog.smithi152:Node is ready 2022-01-06T10:56:30.603 INFO:teuthology.orchestra.run.smithi152.stdout:smithi152.front.sepia.ceph.com 2022-01-06T10:56:30.662 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152 smithi152.front.sepia.ceph.com smithi152 2022-01-06T10:56:31.031 INFO:teuthology.provision.fog.smithi152:Deploy complete! 2022-01-06T10:56:31.181 INFO:teuthology.lock.ops:Checking smithi055.front.sepia.ceph.com 2022-01-06T10:56:31.182 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-06T10:56:31.203 INFO:teuthology.lock.ops:Checking smithi152.front.sepia.ceph.com 2022-01-06T10:56:31.204 INFO:teuthology.lock.ops:New key found. Updating... 2022-01-06T10:56:31.224 INFO:teuthology.lock.ops:Updating [smithi055.front.sepia.ceph.com]: set os type and version on server 2022-01-06T10:56:31.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2022-01-06T10:56:31.476 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2022-01-06T10:56:31.533 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="CentOS Stream" 2022-01-06T10:56:31.533 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="8" 2022-01-06T10:56:31.533 INFO:teuthology.orchestra.run.smithi055.stdout:ID="centos" 2022-01-06T10:56:31.534 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="rhel fedora" 2022-01-06T10:56:31.534 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="8" 2022-01-06T10:56:31.534 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el8" 2022-01-06T10:56:31.534 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-06T10:56:31.535 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2022-01-06T10:56:31.535 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-06T10:56:31.535 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://centos.org/" 2022-01-06T10:56:31.536 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-06T10:56:31.536 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-06T10:56:31.536 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-06T10:56:31.537 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2022-01-06T10:56:31.556 INFO:teuthology.lock.ops:Updating [smithi152.front.sepia.ceph.com]: set os type and version on server 2022-01-06T10:56:31.556 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2022-01-06T10:56:31.743 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2022-01-06T10:56:31.799 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="CentOS Stream" 2022-01-06T10:56:31.799 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="8" 2022-01-06T10:56:31.800 INFO:teuthology.orchestra.run.smithi152.stdout:ID="centos" 2022-01-06T10:56:31.800 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="rhel fedora" 2022-01-06T10:56:31.800 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="8" 2022-01-06T10:56:31.800 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el8" 2022-01-06T10:56:31.801 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="CentOS Stream 8" 2022-01-06T10:56:31.801 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2022-01-06T10:56:31.801 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-01-06T10:56:31.801 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://centos.org/" 2022-01-06T10:56:31.802 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-01-06T10:56:31.802 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-01-06T10:56:31.802 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-01-06T10:56:31.803 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2022-01-06T10:56:31.894 INFO:teuthology.dispatcher.supervisor:Running job 6597702 2022-01-06T10:56:31.895 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_swagner@teuthology --archive /home/teuthworker/archive/swagner-2022-01-06_10:04:35-orch:cephadm-wip-swagner-testing-2022-01-05-1435-distro-default-smithi/6597702 --name swagner-2022-01-06_10:04:35-orch:cephadm-wip-swagner-testing-2022-01-05-1435-distro-default-smithi --description orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/swagner-2022-01-06_10:04:35-orch:cephadm-wip-swagner-testing-2022-01-05-1435-distro-default-smithi/6597702/orig.config.yaml 2022-01-06T10:56:31.898 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/swagner-2022-01-06_10:04:35-orch:cephadm-wip-swagner-testing-2022-01-05-1435-distro-default-smithi/6597702 2022-01-06T10:56:31.899 INFO:teuthology.dispatcher.supervisor:Job PID: 18011 2022-01-06T10:56:31.899 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-01-06T22:57:25.963 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-01-06T22:57:26.154 INFO:teuthology.kill:Killing Pids: {18011} 2022-01-06T22:57:27.345 INFO:teuthology.task.internal:roles: ubuntu@smithi055.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-01-06T22:57:27.346 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-01-06T22:57:27.347 INFO:teuthology.misc:Compressing logs... 2022-01-06T22:57:27.347 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2022-01-06T22:57:27.546 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2022-01-06T22:57:27.598 INFO:teuthology.orchestra.run.smithi055.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-06T22:57:27.791 INFO:teuthology.orchestra.run.smithi152.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-01-06T22:57:27.831 INFO:teuthology.orchestra.run.smithi055.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-06T22:57:27.831 INFO:teuthology.orchestra.run.smithi055.stderr:tar: Error is not recoverable: exiting now 2022-01-06T22:57:27.871 INFO:teuthology.orchestra.run.smithi152.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-01-06T22:57:27.871 INFO:teuthology.orchestra.run.smithi152.stderr:tar: Error is not recoverable: exiting now 2022-01-06T22:57:27.873 INFO:teuthology.misc:Compressing logs... 2022-01-06T22:57:28.488 INFO:teuthology.misc:Compressing logs... 2022-01-06T22:57:56.115 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/7b60a820-6ee0-11ec-8c32-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-01-06T22:57:58.200 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/7b60a820-6ee0-11ec-8c32-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-01-06T22:58:22.049 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/7b60a820-6ee0-11ec-8c32-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-01-06T22:58:23.058 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/7b60a820-6ee0-11ec-8c32-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-01-06T22:58:33.334 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/7b60a820-6ee0-11ec-8c32-001a4aab830c/ceph-mon.smithi152.log: file size changed while zipping 2022-01-06T22:58:40.114 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/7b60a820-6ee0-11ec-8c32-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-01-06T22:58:58.436 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/7b60a820-6ee0-11ec-8c32-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-01-06T22:59:00.791 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/7b60a820-6ee0-11ec-8c32-001a4aab830c/ceph-mon.smithi055.log: file size changed while zipping 2022-01-06T22:59:20.349 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/7b60a820-6ee0-11ec-8c32-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-01-06T22:59:24.660 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/7b60a820-6ee0-11ec-8c32-001a4aab830c/ceph-mgr.smithi152.vryoor.log: file size changed while zipping 2022-01-06T22:59:27.988 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/7b60a820-6ee0-11ec-8c32-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-01-06T23:00:16.704 INFO:teuthology.kill:No teuthology processes running 2022-01-06T23:00:17.894 INFO:teuthology.kill:Nuking machines: ['smithi055', 'smithi152'] 2022-01-06T23:00:18.945 INFO:teuthology.kill:2022-01-06 23:00:18,945.945 INFO:teuthology.nuke:targets: 2022-01-06T23:00:18.946 INFO:teuthology.kill: smithi055.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLb/WiPglPGDYCkXchGswhecDkgVCM7VmtuO5IGPnMrEEBEuYFdC2FIqMFMMNgpplhnAE8nIBIYAog4yw8rGJO/esiodZrBZUnGZH4WB4b7AMoMv365gGhNOpeLJQCMMIxM00QMzF9QjjbSct2hJJ5S2MY1zeE2A/9LMMZCJiFLpzxgnvzo66nLV4v6Qc3ARNHnCa/ZzVFxgOC3zOwHWlct+ik9jbx6MtaPyqvasx1qyh+4/mbWXjSU8LVJOXW4jrbBPMCAhM9Wq1OmrkrOwWgiXw4Oo1igIfM23b0qoWkbtOqsFHXJwK7jH3EiQlxraGU6e9uKhPFYEb/pcB+6QgtiFf7sV2Ys3MOvYqozW1/LI4Ql86Xqox06kY30xBHN/OhJ2eT4U2cqEi2AVIx3ZrcUCblDG1ixexqsnRO+PAdM9pNjt1ZuvHoh7fq+dWc+g44lEKGhs8LJyGJ3bEvOEzkBFLoBcLtlUWagSP8mj15OmGLdaxxaN9j0Rya0mglLc8= 2022-01-06T23:00:18.947 INFO:teuthology.kill: smithi152.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCn87J9v5551oh6GmkIjXDgjNYf27/kV5nZLB/iiUlbVe1d6hifqfnf1csCoHg2YWE0qo7yBTZ8cGHc3SWkdx+aELlMamEsTnPYnYYb+RwpZxY2lHQdNEl/+7XHL+PVgmgenT2sgoJZH46w9gnRh6NUg9aCGmS+qQQaOZcLB2afGs8z4todh+OtFoW2XBUQ6Ui6iS+hYEpMboBPxXta97xluqZur/NBZFUUnuW9fu0nZe9y9c+UOtdlazSORJEaX5QHyhN1/sVxD/dk9FqoayYdC6oqKiUOUsAxKNJgENdm5gqiAFfTIAtSJaVokyLczMEv4dtWeWnSH2t+hIIvqaGZXQ+kIInfYrEOtCYj778AkUwXP/ckL2ffiiFnUFqTKdHwAU7k9v+oAkWO0eS6DW5HAWkIoBMxi0d/U2YxHi15HF2U0V22k/5sGoqoGm5BeRRHa6p/CY4kZxTFX1as7sx7+sDeJGeP0Xkmr7wIAVAoTqspq3MBDCCCRlrKEGHfXbU= 2022-01-06T23:00:18.974 INFO:teuthology.kill:2022-01-06 23:00:18,973.973 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-06T23:00:18.978 INFO:teuthology.kill:2022-01-06 23:00:18,978.978 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-01-06T23:00:19.055 INFO:teuthology.kill:2022-01-06 23:00:19,055.055 INFO:teuthology.orchestra.console:Power off smithi152 2022-01-06T23:00:19.071 INFO:teuthology.kill:2022-01-06 23:00:19,071.071 INFO:teuthology.orchestra.console:Power off smithi055 2022-01-06T23:00:27.433 INFO:teuthology.kill:2022-01-06 23:00:27,432.432 INFO:teuthology.orchestra.console:Power off for smithi152 completed 2022-01-06T23:00:27.437 INFO:teuthology.kill:2022-01-06 23:00:27,437.437 INFO:teuthology.orchestra.console:Power off for smithi055 completed 2022-01-06T23:00:27.586 INFO:teuthology.kill:2022-01-06 23:00:27,584.584 INFO:teuthology.lock.ops:unlocked: smithi152.front.sepia.ceph.com 2022-01-06T23:00:27.588 INFO:teuthology.kill:2022-01-06 23:00:27,588.588 INFO:teuthology.lock.ops:unlocked: smithi055.front.sepia.ceph.com 2022-01-06T23:02:27.904 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-01-06T23:02:28.003 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi055 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-06_15:50:38-rados-wip-yuri8-testing-2022-01-05-1411-distro-default-smithi/6599159 2022-01-06T23:02:28.004 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi152 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-01-06_15:50:38-rados-wip-yuri8-testing-2022-01-05-1411-distro-default-smithi/6599159