2022-02-13T05:43:39.117 INFO:root:teuthology version: 1.1.0-7cebb3f2
2022-02-13T05:43:39.256 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi092.front.sepia.ceph.com' reimaging
2022-02-13T05:43:39.258 INFO:teuthology.lock.ops:Updating [ubuntu@smithi092.front.sepia.ceph.com]: reset os type and version on server
2022-02-13T05:43:39.259 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server
2022-02-13T05:43:39.281 INFO:teuthology.lock.ops:Node 'ubuntu@smithi092.front.sepia.ceph.com' reimaging is complete
2022-02-13T05:43:39.282 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi152.front.sepia.ceph.com' reimaging
2022-02-13T05:43:39.282 INFO:teuthology.lock.ops:Updating [ubuntu@smithi152.front.sepia.ceph.com]: reset os type and version on server
2022-02-13T05:43:39.283 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server
2022-02-13T05:43:39.301 INFO:teuthology.lock.ops:Node 'ubuntu@smithi152.front.sepia.ceph.com' reimaging is complete
2022-02-13T05:43:39.499 INFO:teuthology.provision.fog.smithi092:Scheduling deploy of centos 8.stream
2022-02-13T05:43:39.515 INFO:teuthology.provision.fog.smithi152:Scheduling deploy of centos 8.stream
2022-02-13T05:43:39.907 INFO:teuthology.orchestra.console:Power off smithi152
2022-02-13T05:43:39.922 INFO:teuthology.orchestra.console:Power off smithi092
2022-02-13T05:43:52.392 INFO:teuthology.orchestra.console:Power off for smithi152 completed
2022-02-13T05:43:52.397 INFO:teuthology.orchestra.console:Power off for smithi092 completed
2022-02-13T05:43:52.494 INFO:teuthology.orchestra.console:Power on smithi152
2022-02-13T05:43:52.503 INFO:teuthology.orchestra.console:Power on smithi092
2022-02-13T05:43:56.753 INFO:teuthology.orchestra.console:Power on for smithi152 completed
2022-02-13T05:43:56.854 INFO:teuthology.provision.fog.smithi152:Waiting for deploy to finish
2022-02-13T05:44:00.869 INFO:teuthology.orchestra.console:Power on for smithi092 completed
2022-02-13T05:44:00.970 INFO:teuthology.provision.fog.smithi092:Waiting for deploy to finish
2022-02-13T05:48:07.619 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-13T05:48:27.477 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-13T05:48:32.602 INFO:teuthology.provision.fog.smithi092:Node is ready
2022-02-13T05:48:32.625 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092.front.sepia.ceph.com
2022-02-13T05:48:32.684 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92 smithi092.front.sepia.ceph.com smithi092
2022-02-13T05:48:33.060 INFO:teuthology.provision.fog.smithi092:Deploy complete!
2022-02-13T05:48:52.693 INFO:teuthology.provision.fog.smithi152:Node is ready
2022-02-13T05:48:52.716 INFO:teuthology.orchestra.run.smithi152.stdout:smithi152.front.sepia.ceph.com
2022-02-13T05:48:52.775 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152 smithi152.front.sepia.ceph.com smithi152
2022-02-13T05:48:53.143 INFO:teuthology.provision.fog.smithi152:Deploy complete!
2022-02-13T05:48:53.323 INFO:teuthology.lock.ops:Checking smithi092.front.sepia.ceph.com
2022-02-13T05:48:53.324 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-13T05:48:53.344 INFO:teuthology.lock.ops:Checking smithi152.front.sepia.ceph.com
2022-02-13T05:48:53.345 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-13T05:48:53.364 INFO:teuthology.lock.ops:Updating [smithi092.front.sepia.ceph.com]: set os type and version on server
2022-02-13T05:48:53.364 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com'
2022-02-13T05:48:53.618 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64
2022-02-13T05:48:53.678 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream"
2022-02-13T05:48:53.679 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="8"
2022-02-13T05:48:53.679 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos"
2022-02-13T05:48:53.679 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora"
2022-02-13T05:48:53.679 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="8"
2022-02-13T05:48:53.680 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el8"
2022-02-13T05:48:53.680 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-13T05:48:53.680 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31"
2022-02-13T05:48:53.680 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-13T05:48:53.681 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/"
2022-02-13T05:48:53.681 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-13T05:48:53.681 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-13T05:48:53.682 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-13T05:48:53.683 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server
2022-02-13T05:48:53.701 INFO:teuthology.lock.ops:Updating [smithi152.front.sepia.ceph.com]: set os type and version on server
2022-02-13T05:48:53.702 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com'
2022-02-13T05:48:53.940 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64
2022-02-13T05:48:53.998 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="CentOS Stream"
2022-02-13T05:48:53.999 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="8"
2022-02-13T05:48:53.999 INFO:teuthology.orchestra.run.smithi152.stdout:ID="centos"
2022-02-13T05:48:53.999 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="rhel fedora"
2022-02-13T05:48:53.999 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="8"
2022-02-13T05:48:54.000 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el8"
2022-02-13T05:48:54.000 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-13T05:48:54.000 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31"
2022-02-13T05:48:54.000 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-13T05:48:54.001 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://centos.org/"
2022-02-13T05:48:54.001 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-13T05:48:54.001 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-13T05:48:54.001 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-13T05:48:54.002 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server
2022-02-13T05:48:54.098 INFO:teuthology.dispatcher.supervisor:Running job 6679059
2022-02-13T05:48:54.098 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-02-13_03:31:02-rados-pacific-distro-default-smithi/6679059 --name teuthology-2022-02-13_03:31:02-rados-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/teuthology-2022-02-13_03:31:02-rados-pacific-distro-default-smithi/6679059/orig.config.yaml
2022-02-13T05:48:54.102 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-02-13_03:31:02-rados-pacific-distro-default-smithi/6679059
2022-02-13T05:48:54.103 INFO:teuthology.dispatcher.supervisor:Job PID: 16943
2022-02-13T05:48:54.103 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-13T12:19:23.879 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-13T12:19:24.063 INFO:teuthology.kill:Killing Pids: {16943}
2022-02-13T12:19:25.273 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-02-13T12:19:25.274 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-02-13T12:19:25.274 INFO:teuthology.misc:Compressing logs...
2022-02-13T12:19:25.275 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com'
2022-02-13T12:19:25.467 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com'
2022-02-13T12:19:25.520 INFO:teuthology.orchestra.run.smithi092.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-13T12:19:25.708 INFO:teuthology.orchestra.run.smithi152.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-13T12:19:25.750 INFO:teuthology.orchestra.run.smithi092.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-13T12:19:25.750 INFO:teuthology.orchestra.run.smithi092.stderr:tar: Error is not recoverable: exiting now
2022-02-13T12:19:25.786 INFO:teuthology.orchestra.run.smithi152.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-13T12:19:25.787 INFO:teuthology.orchestra.run.smithi152.stderr:tar: Error is not recoverable: exiting now
2022-02-13T12:19:25.788 INFO:teuthology.misc:Compressing logs...
2022-02-13T12:19:26.087 INFO:teuthology.misc:Compressing logs...
2022-02-13T12:19:29.630 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/9273f9d0-8c91-11ec-8c35-001a4aab830c/ceph-mon.smithi152.log: file size changed while zipping
2022-02-13T12:19:42.283 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/9273f9d0-8c91-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-02-13T12:19:46.910 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/9273f9d0-8c91-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-02-13T12:19:54.096 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/9273f9d0-8c91-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-02-13T12:20:05.258 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/9273f9d0-8c91-11ec-8c35-001a4aab830c/ceph-mon.smithi092.log: file size changed while zipping
2022-02-13T12:20:06.835 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/9273f9d0-8c91-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-02-13T12:20:09.730 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/9273f9d0-8c91-11ec-8c35-001a4aab830c/ceph-mgr.smithi092.vxukcp.log: file size changed while zipping
2022-02-13T12:20:21.450 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/9273f9d0-8c91-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-02-13T12:20:27.774 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/9273f9d0-8c91-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-02-13T12:20:37.147 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/9273f9d0-8c91-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-02-13T12:20:45.024 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/9273f9d0-8c91-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-02-13T12:21:15.697 INFO:teuthology.kill:No teuthology processes running
2022-02-13T12:21:16.848 INFO:teuthology.kill:Nuking machines: ['smithi092', 'smithi152']
2022-02-13T12:21:17.962 INFO:teuthology.kill:2022-02-13 12:21:17,962.962 INFO:teuthology.nuke:targets:
2022-02-13T12:21:17.964 INFO:teuthology.kill:  smithi092.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGl7Q8vY4sOo4itqcOqtCd/lML7zEvE4ywkrMqYHtbcNdDJiBA0Htd9/v03eg2Dch1MLBFh4QHGlnk99uML8iYlXrZSJUEu8XtBMKY6Rf0TlnWoftwFIQWS4wRJ0b2VCElunagUKv5X/dwt4KHVi3SZDZzsSThkgaRwDz2Yuo6ctX+aWIR20/xyIr2PF/UuKQOmqGVOdSYWBcRu/XvMXjLZq7zXVv2mVE4ymmdDMRf2q87dFNWuWW/N24OAZfHgE9lHAXl8X6QINWlFXVERqSwhcnARwNCI0Rm+tJcs8AFq9kfakpyGM3qxeca6kA6UO+04wak6LDBIjbCq15FDyjk8Po/CG9EvrfVOWJe+UpmrVKKda/hFGW7fVOfLnBeo26tUopXQ4vxnCJuWoJFXv1RT04W8bW5Wl8KzFVycVA7AKaiFYXfiN0ZAqb1eVJhUDgjaO/uqwlE3/0dPvw8OgTax4WK2WhdEuuXFMIFJeWolv9tEu6qLZi6Dllcilshe28=
2022-02-13T12:21:17.964 INFO:teuthology.kill:  smithi152.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqH1yCVhCcB5FSNH4q5bXI6sK69A8MXUqBM/NXK9ZZRnw9NZ1uCp/ZkwNtw8pUZiOtIJhEzvIErlePW5DP4ZGMQJaUT33d8gO2k7UsgBmXAO6wlviVRY+25iBaWcasILoq4LKlsiltKLNAvf2TNkWm3qUoZHgSyRg4T9fK8OMDWDcjLlaKqqynuZL74UmP1hE3jhDiN9ZcVRBjlUZGmTrn1R+fr/k3uSIkw1pY71X1jVzRuIqi298eNQ7Y96Pz26lMej+gs3Bki31cRyCWnTI82oSseoW9GlqXJ+99RHznkzjmaKs9W6OwHEXnCqAnfqUoVAO59Itam/457XpO0uAs06qvGCerRM+tYnIWORhiGzOkh3PK972EXVHpxRGA118NjvOnExzeybpbymANTYHoUFQg8Rr0y/rvJAfyGcj6f/Bp6E5Cje2OUStW/WAbX46mLQ2h56ULR/srk3Z9Wy8ADpL69tak1CJCYEzy5PnyCKvhTMc47QeBbuIVW2OLadc=
2022-02-13T12:21:17.994 INFO:teuthology.kill:2022-02-13 12:21:17,994.994 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-13T12:21:17.998 INFO:teuthology.kill:2022-02-13 12:21:17,998.998 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-13T12:21:18.077 INFO:teuthology.kill:2022-02-13 12:21:18,077.077 INFO:teuthology.orchestra.console:Power off smithi092
2022-02-13T12:21:18.091 INFO:teuthology.kill:2022-02-13 12:21:18,091.091 INFO:teuthology.orchestra.console:Power off smithi152
2022-02-13T12:21:30.574 INFO:teuthology.kill:2022-02-13 12:21:30,573.573 INFO:teuthology.orchestra.console:Power off for smithi092 completed
2022-02-13T12:21:30.579 INFO:teuthology.kill:2022-02-13 12:21:30,579.579 INFO:teuthology.orchestra.console:Power off for smithi152 completed
2022-02-13T12:21:30.727 INFO:teuthology.kill:2022-02-13 12:21:30,726.726 INFO:teuthology.lock.ops:unlocked: smithi092.front.sepia.ceph.com
2022-02-13T12:21:30.728 INFO:teuthology.kill:2022-02-13 12:21:30,727.727 INFO:teuthology.lock.ops:unlocked: smithi152.front.sepia.ceph.com
2022-02-13T12:23:31.057 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-13T12:23:31.156 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi092 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-02-13_07:01:03-rados-master-distro-default-smithi/6679526
2022-02-13T12:23:31.156 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi152 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-02-13_07:01:03-rados-master-distro-default-smithi/6679525