2022-01-23T15:45:14.251 INFO:root:teuthology version: 1.1.0-75f1bcd2
2022-01-23T15:45:14.385 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi096.front.sepia.ceph.com' reimaging
2022-01-23T15:45:14.386 INFO:teuthology.lock.ops:Updating [ubuntu@smithi096.front.sepia.ceph.com]: reset os type and version on server
2022-01-23T15:45:14.386 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server
2022-01-23T15:45:14.406 INFO:teuthology.lock.ops:Node 'ubuntu@smithi096.front.sepia.ceph.com' reimaging is complete
2022-01-23T15:45:14.406 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi125.front.sepia.ceph.com' reimaging
2022-01-23T15:45:14.407 INFO:teuthology.lock.ops:Updating [ubuntu@smithi125.front.sepia.ceph.com]: reset os type and version on server
2022-01-23T15:45:14.407 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server
2022-01-23T15:45:14.427 INFO:teuthology.lock.ops:Node 'ubuntu@smithi125.front.sepia.ceph.com' reimaging is complete
2022-01-23T15:45:14.619 INFO:teuthology.provision.fog.smithi096:Scheduling deploy of centos 8.3
2022-01-23T15:45:14.643 INFO:teuthology.provision.fog.smithi125:Scheduling deploy of centos 8.3
2022-01-23T15:45:14.965 INFO:teuthology.orchestra.console:Power off smithi096
2022-01-23T15:45:14.981 INFO:teuthology.orchestra.console:Power off smithi125
2022-01-23T15:45:15.110 INFO:teuthology.orchestra.console:Power off for smithi096 completed
2022-01-23T15:45:15.122 INFO:teuthology.orchestra.console:Power off for smithi125 completed
2022-01-23T15:45:15.212 INFO:teuthology.orchestra.console:Power on smithi096
2022-01-23T15:45:15.224 INFO:teuthology.orchestra.console:Power on smithi125
2022-01-23T15:45:19.478 INFO:teuthology.orchestra.console:Power on for smithi125 completed
2022-01-23T15:45:19.580 INFO:teuthology.provision.fog.smithi125:Waiting for deploy to finish
2022-01-23T15:45:23.585 INFO:teuthology.orchestra.console:Power on for smithi096 completed
2022-01-23T15:45:23.687 INFO:teuthology.provision.fog.smithi096:Waiting for deploy to finish
2022-01-23T15:49:48.799 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-23T15:49:49.096 INFO:teuthology.provision.fog.smithi125:Node is ready
2022-01-23T15:49:49.111 INFO:teuthology.orchestra.run.smithi125.stdout:smithi125.front.sepia.ceph.com
2022-01-23T15:49:49.165 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.125 smithi125.front.sepia.ceph.com smithi125
2022-01-23T15:49:49.478 INFO:teuthology.provision.fog.smithi125:Deploy complete!
2022-01-23T15:49:55.497 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-23T15:49:55.795 INFO:teuthology.provision.fog.smithi096:Node is ready
2022-01-23T15:49:55.811 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096.front.sepia.ceph.com
2022-01-23T15:49:55.866 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96 smithi096.front.sepia.ceph.com smithi096
2022-01-23T15:49:56.192 INFO:teuthology.provision.fog.smithi096:Deploy complete!
2022-01-23T15:49:56.341 INFO:teuthology.lock.ops:Checking smithi096.front.sepia.ceph.com
2022-01-23T15:49:56.342 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-23T15:49:56.362 INFO:teuthology.lock.ops:Checking smithi125.front.sepia.ceph.com
2022-01-23T15:49:56.362 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-23T15:49:56.382 INFO:teuthology.lock.ops:Updating [smithi096.front.sepia.ceph.com]: set os type and version on server
2022-01-23T15:49:56.383 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com'
2022-01-23T15:49:56.634 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64
2022-01-23T15:49:56.688 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Linux"
2022-01-23T15:49:56.688 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="8"
2022-01-23T15:49:56.689 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos"
2022-01-23T15:49:56.689 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora"
2022-01-23T15:49:56.689 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="8"
2022-01-23T15:49:56.689 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el8"
2022-01-23T15:49:56.690 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Linux 8"
2022-01-23T15:49:56.690 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31"
2022-01-23T15:49:56.690 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-01-23T15:49:56.690 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/"
2022-01-23T15:49:56.691 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2022-01-23T15:49:56.691 INFO:teuthology.orchestra.run.smithi096.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2022-01-23T15:49:56.691 INFO:teuthology.orchestra.run.smithi096.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2022-01-23T15:49:56.692 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server
2022-01-23T15:49:56.710 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: set os type and version on server
2022-01-23T15:49:56.710 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com'
2022-01-23T15:49:56.981 INFO:teuthology.orchestra.run.smithi125.stdout:x86_64
2022-01-23T15:49:57.036 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="CentOS Linux"
2022-01-23T15:49:57.037 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="8"
2022-01-23T15:49:57.037 INFO:teuthology.orchestra.run.smithi125.stdout:ID="centos"
2022-01-23T15:49:57.037 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE="rhel fedora"
2022-01-23T15:49:57.037 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="8"
2022-01-23T15:49:57.038 INFO:teuthology.orchestra.run.smithi125.stdout:PLATFORM_ID="platform:el8"
2022-01-23T15:49:57.038 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="CentOS Linux 8"
2022-01-23T15:49:57.038 INFO:teuthology.orchestra.run.smithi125.stdout:ANSI_COLOR="0;31"
2022-01-23T15:49:57.038 INFO:teuthology.orchestra.run.smithi125.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-01-23T15:49:57.039 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://centos.org/"
2022-01-23T15:49:57.039 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2022-01-23T15:49:57.039 INFO:teuthology.orchestra.run.smithi125.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2022-01-23T15:49:57.039 INFO:teuthology.orchestra.run.smithi125.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2022-01-23T15:49:57.040 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server
2022-01-23T15:49:57.141 INFO:teuthology.dispatcher.supervisor:Running job 6635943
2022-01-23T15:49:57.142 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_75f1bcd237a8b6022305cd53ba87e106d45aa665/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-01-23_03:31:03-rados-pacific-distro-default-smithi/6635943 --name teuthology-2022-01-23_03:31:03-rados-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.3_container_tools_3.0 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/teuthology-2022-01-23_03:31:03-rados-pacific-distro-default-smithi/6635943/orig.config.yaml
2022-01-23T15:49:57.146 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-01-23_03:31:03-rados-pacific-distro-default-smithi/6635943
2022-01-23T15:49:57.146 INFO:teuthology.dispatcher.supervisor:Job PID: 16215
2022-01-23T15:49:57.147 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-01-23T22:20:26.155 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-01-23T22:20:26.374 INFO:teuthology.kill:Killing Pids: {16215}
2022-01-23T22:20:27.606 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-01-23T22:20:27.607 INFO:teuthology.task.internal:roles: ubuntu@smithi125.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-01-23T22:20:27.608 INFO:teuthology.misc:Compressing logs...
2022-01-23T22:20:27.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com'
2022-01-23T22:20:27.822 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com'
2022-01-23T22:20:27.868 INFO:teuthology.orchestra.run.smithi096.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-01-23T22:20:28.070 INFO:teuthology.orchestra.run.smithi125.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-01-23T22:20:28.111 INFO:teuthology.orchestra.run.smithi096.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-01-23T22:20:28.112 INFO:teuthology.orchestra.run.smithi096.stderr:tar: Error is not recoverable: exiting now
2022-01-23T22:20:28.150 INFO:teuthology.orchestra.run.smithi125.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-01-23T22:20:28.150 INFO:teuthology.orchestra.run.smithi125.stderr:tar: Error is not recoverable: exiting now
2022-01-23T22:20:28.151 INFO:teuthology.misc:Compressing logs...
2022-01-23T22:20:28.467 INFO:teuthology.misc:Compressing logs...
2022-01-23T22:20:49.691 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /var/log/ceph/60c3eb84-7c65-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-01-23T22:20:52.468 INFO:teuthology.orchestra.run.smithi125.stderr:gzip: /var/log/ceph/60c3eb84-7c65-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-01-23T22:20:54.471 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /var/log/ceph/60c3eb84-7c65-11ec-8c35-001a4aab830c/ceph-mgr.smithi096.drorep.log: file size changed while zipping
2022-01-23T22:21:06.275 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /var/log/ceph/60c3eb84-7c65-11ec-8c35-001a4aab830c/ceph-mon.smithi096.log: file size changed while zipping
2022-01-23T22:21:12.896 INFO:teuthology.orchestra.run.smithi125.stderr:gzip: /var/log/ceph/60c3eb84-7c65-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-01-23T22:21:24.252 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /var/log/ceph/60c3eb84-7c65-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-01-23T22:21:31.186 INFO:teuthology.orchestra.run.smithi125.stderr:gzip: /var/log/ceph/60c3eb84-7c65-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-01-23T22:21:34.522 INFO:teuthology.orchestra.run.smithi125.stderr:gzip: /var/log/ceph/60c3eb84-7c65-11ec-8c35-001a4aab830c/ceph-mon.smithi125.log: file size changed while zipping
2022-01-23T22:21:35.945 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /var/log/ceph/60c3eb84-7c65-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-01-23T22:21:51.423 INFO:teuthology.orchestra.run.smithi125.stderr:gzip: /var/log/ceph/60c3eb84-7c65-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-01-23T22:21:56.979 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /var/log/ceph/60c3eb84-7c65-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-01-23T22:22:33.309 INFO:teuthology.orchestra.run.smithi125.stderr:tar: .: file changed as we read it
2022-01-23T22:22:33.373 INFO:teuthology.kill:No teuthology processes running
2022-01-23T22:22:34.499 INFO:teuthology.kill:Nuking machines: ['smithi096', 'smithi125']
2022-01-23T22:22:35.532 INFO:teuthology.kill:2022-01-23 22:22:35,531.531 INFO:teuthology.nuke:targets:
2022-01-23T22:22:35.533 INFO:teuthology.kill:  smithi096.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcBKFgCf9V74JlK0QEp1mphteCLp2GaLmoPgInGMA5BXMXweDZSCZnZISLkr3j4Z4sI5as9P8k/er+TWusbRuTuDwD59Xo2J5+akrYhRu4pNeOHL8P/IctaOlPhIk7YHygvCMJ7nmB5Gv1KtemMQioiscRkZMJlF2yiJYhqS9p9W7XrNPix2T2+DEcVuYl9iR6u5bK6bBl1F6y0/MeMEAEKoJn6gwTPPeSLtjq9PAcMBdDesQsyl6+YVB30Q0QhVpKmbSzmr/3kHr7eQcm5LSe5a7cvKsYuOY7XYt7gPc7ErWSZBP2HWw9EYHDI24SNs7tCFD50LbWOf/StABafzuB2hD9ueKAbL06Ote4J71Zbw23ctnbFITwNeXotbfHca80GtIhlZedtXUucguf6T2yEo9QV8MGQreHYwWBmgZYnNJP1gx58QeQ9/jyjkHiZPCA4d6/YNM2QuAo3yy99tfbSAVHcLi/hzMzF2oz9bgVMOyiTw/pOt2wCeV/3nbB+9E=
2022-01-23T22:22:35.533 INFO:teuthology.kill:  smithi125.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDN7QYpyawMfcIRvnkMvY3HyALi7zNX/VTZ/mRdEHFV07ffeVU+sv1tkoqZ7HAwclGBjS/RZu587M3OdzmPh9ljyHMWqUNfP6R17lsP4NPPKXgfGc2lHCGYh+qX9uwhc3HZYc4+A5B7PG7hTwz9Hbr8/RerdenW5oi3nQOfBA40cfAqWkVBcJS361lzgZna2MT4o1Wl2hvWlMWEFVieQbP5LQahOLUEOFeDFEyY2KJXRvDQUeH2gUSIiAfZVIq59mU6rTUdOW1Yl4+6NvTLnBAStDUon4Ue6V5Huxd43XqiSl8V+ysfQwzzjY8IdrQfZq8yrrvxSDU4SK4hIc0MQ+sEYRBaR38F2NCR3nr3uq8X3FfUIh7pmCxzrn4GasZBRzKSGRnsGxrGO0jDdaalhSdyO5rHL4Bd6PAcEFjhH3+9NglLMU19qQkWNBMP+KKoZKsB6Uj7V9RwfrOFuKgvXuedmUcsnNVlo8lOEsThZRZYAYR5/MfFxLzloQ1Jg/fLdoE=
2022-01-23T22:22:35.562 INFO:teuthology.kill:2022-01-23 22:22:35,562.562 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-23T22:22:35.564 INFO:teuthology.kill:2022-01-23 22:22:35,563.563 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-23T22:22:35.642 INFO:teuthology.kill:2022-01-23 22:22:35,642.642 INFO:teuthology.orchestra.console:Power off smithi096
2022-01-23T22:22:35.654 INFO:teuthology.kill:2022-01-23 22:22:35,654.654 INFO:teuthology.orchestra.console:Power off smithi125
2022-01-23T22:22:48.126 INFO:teuthology.kill:2022-01-23 22:22:48,125.125 INFO:teuthology.orchestra.console:Power off for smithi096 completed
2022-01-23T22:22:48.132 INFO:teuthology.kill:2022-01-23 22:22:48,131.131 INFO:teuthology.orchestra.console:Power off for smithi125 completed
2022-01-23T22:22:48.279 INFO:teuthology.kill:2022-01-23 22:22:48,279.279 INFO:teuthology.lock.ops:unlocked: smithi096.front.sepia.ceph.com
2022-01-23T22:22:48.284 INFO:teuthology.kill:2022-01-23 22:22:48,283.283 INFO:teuthology.lock.ops:unlocked: smithi125.front.sepia.ceph.com
2022-01-23T22:24:48.618 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-01-23T22:24:48.717 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi096 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-20_06:00:02-rbd-octopus-distro-default-smithi/6628606
2022-01-23T22:24:48.718 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi125 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-20_06:00:02-rbd-octopus-distro-default-smithi/6628606