2022-04-10T17:29:38.885 INFO:root:teuthology version: 1.1.0-a347b2ac
2022-04-10T17:29:39.007 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi120.front.sepia.ceph.com' reimaging
2022-04-10T17:29:39.008 INFO:teuthology.lock.ops:Updating [ubuntu@smithi120.front.sepia.ceph.com]: reset os type and version on server
2022-04-10T17:29:39.008 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server
2022-04-10T17:29:39.026 INFO:teuthology.lock.ops:Node 'ubuntu@smithi120.front.sepia.ceph.com' reimaging is complete
2022-04-10T17:29:39.026 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi149.front.sepia.ceph.com' reimaging
2022-04-10T17:29:39.027 INFO:teuthology.lock.ops:Updating [ubuntu@smithi149.front.sepia.ceph.com]: reset os type and version on server
2022-04-10T17:29:39.027 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server
2022-04-10T17:29:39.050 INFO:teuthology.lock.ops:Node 'ubuntu@smithi149.front.sepia.ceph.com' reimaging is complete
2022-04-10T17:29:39.241 INFO:teuthology.provision.fog.smithi120:Scheduling deploy of centos 8.stream
2022-04-10T17:29:39.269 INFO:teuthology.provision.fog.smithi149:Scheduling deploy of centos 8.stream
2022-04-10T17:29:39.589 INFO:teuthology.orchestra.console:Power off smithi120
2022-04-10T17:29:39.626 INFO:teuthology.orchestra.console:Power off smithi149
2022-04-10T17:29:47.961 INFO:teuthology.orchestra.console:Power off for smithi120 completed
2022-04-10T17:29:47.986 INFO:teuthology.orchestra.console:Power off for smithi149 completed
2022-04-10T17:29:48.063 INFO:teuthology.orchestra.console:Power on smithi120
2022-04-10T17:29:48.089 INFO:teuthology.orchestra.console:Power on smithi149
2022-04-10T17:29:56.429 INFO:teuthology.orchestra.console:Power on for smithi120 completed
2022-04-10T17:29:56.443 INFO:teuthology.orchestra.console:Power on for smithi149 completed
2022-04-10T17:29:56.531 INFO:teuthology.provision.fog.smithi120:Waiting for deploy to finish
2022-04-10T17:29:56.544 INFO:teuthology.provision.fog.smithi149:Waiting for deploy to finish
2022-04-10T17:34:25.634 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-10T17:34:25.916 INFO:teuthology.provision.fog.smithi120:Node is ready
2022-04-10T17:34:25.934 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120.front.sepia.ceph.com
2022-04-10T17:34:25.992 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120 smithi120.front.sepia.ceph.com smithi120
2022-04-10T17:34:26.349 INFO:teuthology.provision.fog.smithi120:Deploy complete!
2022-04-10T17:34:34.645 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-10T17:34:34.914 INFO:teuthology.provision.fog.smithi149:Node is ready
2022-04-10T17:34:34.930 INFO:teuthology.orchestra.run.smithi149.stdout:smithi149.front.sepia.ceph.com
2022-04-10T17:34:34.987 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.15.149 smithi149.front.sepia.ceph.com smithi149
2022-04-10T17:34:35.319 INFO:teuthology.provision.fog.smithi149:Deploy complete!
2022-04-10T17:34:35.466 INFO:teuthology.lock.ops:Checking smithi120.front.sepia.ceph.com
2022-04-10T17:34:35.467 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-10T17:34:35.485 INFO:teuthology.lock.ops:Checking smithi149.front.sepia.ceph.com
2022-04-10T17:34:35.486 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-10T17:34:35.504 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: set os type and version on server
2022-04-10T17:34:35.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com'
2022-04-10T17:34:35.700 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64
2022-04-10T17:34:35.758 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="CentOS Stream"
2022-04-10T17:34:35.758 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="8"
2022-04-10T17:34:35.758 INFO:teuthology.orchestra.run.smithi120.stdout:ID="centos"
2022-04-10T17:34:35.759 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="rhel fedora"
2022-04-10T17:34:35.759 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="8"
2022-04-10T17:34:35.759 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el8"
2022-04-10T17:34:35.759 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="CentOS Stream 8"
2022-04-10T17:34:35.760 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31"
2022-04-10T17:34:35.760 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-04-10T17:34:35.760 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://centos.org/"
2022-04-10T17:34:35.760 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-10T17:34:35.761 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-10T17:34:35.761 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-04-10T17:34:35.762 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server
2022-04-10T17:34:35.782 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: set os type and version on server
2022-04-10T17:34:35.783 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com'
2022-04-10T17:34:35.966 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64
2022-04-10T17:34:36.022 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream"
2022-04-10T17:34:36.022 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="8"
2022-04-10T17:34:36.023 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos"
2022-04-10T17:34:36.023 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora"
2022-04-10T17:34:36.023 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="8"
2022-04-10T17:34:36.024 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el8"
2022-04-10T17:34:36.024 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 8"
2022-04-10T17:34:36.024 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31"
2022-04-10T17:34:36.025 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-04-10T17:34:36.025 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/"
2022-04-10T17:34:36.025 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-10T17:34:36.025 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-10T17:34:36.026 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-04-10T17:34:36.026 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server
2022-04-10T17:34:36.129 INFO:teuthology.dispatcher.supervisor:Running job 6785111
2022-04-10T17:34:36.129 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a347b2ac5400a43ba76a678399b1deda7dae1a6c/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-04-10_03:31:02-rados-pacific-distro-default-smithi/6785111 --name teuthology-2022-04-10_03:31:02-rados-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/teuthology-2022-04-10_03:31:02-rados-pacific-distro-default-smithi/6785111/orig.config.yaml
2022-04-10T17:34:36.133 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-04-10_03:31:02-rados-pacific-distro-default-smithi/6785111
2022-04-10T17:34:36.133 INFO:teuthology.dispatcher.supervisor:Job PID: 20166
2022-04-10T17:34:36.134 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-04-11T00:05:03.440 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-04-11T00:05:03.590 INFO:teuthology.kill:Killing Pids: {20166}
2022-04-11T00:05:04.775 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-04-11T00:05:04.776 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-04-11T00:05:04.777 INFO:teuthology.misc:Compressing logs...
2022-04-11T00:05:04.777 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com'
2022-04-11T00:05:04.964 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com'
2022-04-11T00:05:05.013 INFO:teuthology.orchestra.run.smithi120.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-04-11T00:05:05.354 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-04-11T00:05:05.395 INFO:teuthology.orchestra.run.smithi120.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-04-11T00:05:05.396 INFO:teuthology.orchestra.run.smithi120.stderr:tar: Error is not recoverable: exiting now
2022-04-11T00:05:05.445 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-04-11T00:05:05.446 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now
2022-04-11T00:05:05.455 INFO:teuthology.misc:Compressing logs...
2022-04-11T00:05:05.694 INFO:teuthology.misc:Compressing logs...
2022-04-11T00:05:19.917 INFO:teuthology.orchestra.run.smithi120.stderr:gzip: /var/log/ceph/ea7972c4-b8f4-11ec-8c37-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-04-11T00:05:34.394 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/ea7972c4-b8f4-11ec-8c37-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-04-11T00:05:35.956 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/ea7972c4-b8f4-11ec-8c37-001a4aab830c/ceph-mon.smithi149.log: file size changed while zipping
2022-04-11T00:05:41.479 INFO:teuthology.orchestra.run.smithi120.stderr:gzip: /var/log/ceph/ea7972c4-b8f4-11ec-8c37-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-04-11T00:05:41.746 INFO:teuthology.orchestra.run.smithi120.stderr:gzip: /var/log/ceph/ea7972c4-b8f4-11ec-8c37-001a4aab830c/ceph-mgr.smithi120.xlzmxn.log: file size changed while zipping
2022-04-11T00:05:55.934 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/ea7972c4-b8f4-11ec-8c37-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-04-11T00:06:05.245 INFO:teuthology.orchestra.run.smithi120.stderr:gzip: /var/log/ceph/ea7972c4-b8f4-11ec-8c37-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-04-11T00:06:16.988 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/ea7972c4-b8f4-11ec-8c37-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-04-11T00:06:29.654 INFO:teuthology.orchestra.run.smithi120.stderr:gzip: /var/log/ceph/ea7972c4-b8f4-11ec-8c37-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-04-11T00:06:34.472 INFO:teuthology.orchestra.run.smithi120.stderr:gzip: /var/log/ceph/ea7972c4-b8f4-11ec-8c37-001a4aab830c/ceph-mon.smithi120.log: file size changed while zipping
2022-04-11T00:06:41.618 INFO:teuthology.orchestra.run.smithi149.stderr:gzip: /var/log/ceph/ea7972c4-b8f4-11ec-8c37-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-04-11T00:07:18.123 INFO:teuthology.kill:No teuthology processes running
2022-04-11T00:07:19.340 INFO:teuthology.kill:Nuking machines: ['smithi120', 'smithi149']
2022-04-11T00:07:20.404 INFO:teuthology.kill:2022-04-11 00:07:20,399.399 INFO:teuthology.nuke:targets:
2022-04-11T00:07:20.405 INFO:teuthology.kill:  smithi120.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfGWkvKoT28so4xXUaky4d/d76unqNb77K5TwgbSElZfBovHf7vQhHB05xQbQL6V7y/BRjImLcoPl38SjvHFKwDvlvwM1rSY7oBjWR/rchMwzc10/lGhfLmWk9swpGVc3w20M2Efsn7geoCkYb/eYJfHSs/gNemJaojG5oWZ2MGvBWXkI5WF9wvq2Dx+kq0yFOCNm90Ayr/WlO4hsMjvfsCQ37UHrqZvjjP/f9OsR3tn2tQhUQA/Q9KNnRKQWRKl2QAAfnHuv7kZfjE2iFAXiNmGblQxRancSpPQA8rH+1eIHz7kbp4rhXW8Uh/miozs7IGi8hQHI7mqhPzJQ02HKNbPb+ciYeFLp7IDWNmtqKPAT/msa8gZv0E/Nl+OESKjTEyjM/fE+47uj3dFcTGXZ7PUWL2SMB7TCpE3t0FFT3bqHwj7e1N/43dT7lB9wFb2kytxJmX9NGN312k3EqNK9KyAAEJwszX7a7j4NMdGYaDurWXBXMuJZOMqbpLuCbgw0=
2022-04-11T00:07:20.405 INFO:teuthology.kill:  smithi149.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHWG00EcsScbXLQhPYk9yuN5BpD/Ps5OINR4Lkn5/E2FYLPragP1ZCxU+qPa4HSKrR5e+FaPol+QoThK5b9bcuMwE0A8FFHSdnfvWnHpekXgvNkfrrn3n2XkmSHJ67M29vNSVEyTjlQw+p3+RcWu050sxBF79c4hLNEaY+/vjeekoyVeJ03iUcLZM2Z40TMg6yqpagWJW4UpUPnpCuGkvTxjCOY3G3ppIQ1l9YJqYcla3RxjQwASBNUx8MWLOzS/0E+6MiYK53j/92G3cPvsq5t4SUOM0ZfdlKBEs3MECPQvQ9i+aQUbke2a2+5zB/ZihbgoTi9n1DgCjSBAjPks5zQbZ5rYpMOFu1dAyEoVKDogtXRGNrrnaT8SD0jVtnmY341ydKwz95HgFrTHt6564G5UdwQnSXKREh4oAYsveW88wCQ+4deQVtBcgmEX9J/KbrNOhVEX9OpWS2xG7ZYTwo9XOOonFDDriKjMrGPTlS0PtRsxm7umz0K/CPi6pfoHU=
2022-04-11T00:07:20.431 INFO:teuthology.kill:2022-04-11 00:07:20,431.431 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-11T00:07:20.434 INFO:teuthology.kill:2022-04-11 00:07:20,434.434 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-11T00:07:20.516 INFO:teuthology.kill:2022-04-11 00:07:20,515.515 INFO:teuthology.orchestra.console:Power off smithi120
2022-04-11T00:07:20.533 INFO:teuthology.kill:2022-04-11 00:07:20,533.533 INFO:teuthology.orchestra.console:Power off smithi149
2022-04-11T00:07:28.899 INFO:teuthology.kill:2022-04-11 00:07:28,898.898 INFO:teuthology.orchestra.console:Power off for smithi120 completed
2022-04-11T00:07:28.905 INFO:teuthology.kill:2022-04-11 00:07:28,905.905 INFO:teuthology.orchestra.console:Power off for smithi149 completed
2022-04-11T00:07:29.059 INFO:teuthology.kill:2022-04-11 00:07:29,059.059 INFO:teuthology.lock.ops:unlocked: smithi120.front.sepia.ceph.com
2022-04-11T00:07:29.063 INFO:teuthology.kill:2022-04-11 00:07:29,062.062 INFO:teuthology.lock.ops:unlocked: smithi149.front.sepia.ceph.com
2022-04-11T00:09:29.371 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-04-11T00:09:29.463 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi120 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-04-09_03:57:03-perf-basic-quincy-distro-default-smithi/6783756
2022-04-11T00:09:29.463 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi149 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-04-09_03:57:03-perf-basic-quincy-distro-default-smithi/6783755