2021-12-02T21:07:03.802 INFO:root:teuthology version: 1.1.0-844d1377
2021-12-02T21:07:03.927 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi017.front.sepia.ceph.com' reimaging
2021-12-02T21:07:03.928 INFO:teuthology.lock.ops:Updating [ubuntu@smithi017.front.sepia.ceph.com]: reset os type and version on server
2021-12-02T21:07:03.928 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server
2021-12-02T21:07:03.953 INFO:teuthology.lock.ops:Node 'ubuntu@smithi017.front.sepia.ceph.com' reimaging is complete
2021-12-02T21:07:03.953 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi168.front.sepia.ceph.com' reimaging
2021-12-02T21:07:03.953 INFO:teuthology.lock.ops:Updating [ubuntu@smithi168.front.sepia.ceph.com]: reset os type and version on server
2021-12-02T21:07:03.954 INFO:teuthology.lock.ops:Updating smithi168.front.sepia.ceph.com on lock server
2021-12-02T21:07:03.975 INFO:teuthology.lock.ops:Node 'ubuntu@smithi168.front.sepia.ceph.com' reimaging is complete
2021-12-02T21:07:04.187 INFO:teuthology.provision.fog.smithi017:Scheduling deploy of centos 8.stream
2021-12-02T21:07:04.251 INFO:teuthology.provision.fog.smithi168:Scheduling deploy of centos 8.stream
2021-12-02T21:07:04.804 INFO:teuthology.orchestra.console:Power off smithi017
2021-12-02T21:07:04.850 INFO:teuthology.orchestra.console:Power off smithi168
2021-12-02T21:07:13.189 INFO:teuthology.orchestra.console:Power off for smithi017 completed
2021-12-02T21:07:13.220 INFO:teuthology.orchestra.console:Power off for smithi168 completed
2021-12-02T21:07:13.291 INFO:teuthology.orchestra.console:Power on smithi017
2021-12-02T21:07:13.321 INFO:teuthology.orchestra.console:Power on smithi168
2021-12-02T21:07:21.663 INFO:teuthology.orchestra.console:Power on for smithi017 completed
2021-12-02T21:07:21.683 INFO:teuthology.orchestra.console:Power on for smithi168 completed
2021-12-02T21:07:21.765 INFO:teuthology.provision.fog.smithi017:Waiting for deploy to finish
2021-12-02T21:07:21.785 INFO:teuthology.provision.fog.smithi168:Waiting for deploy to finish
2021-12-02T21:09:53.302 ERROR:teuthology.orchestra.connection:Error authenticating with smithi168.front.sepia.ceph.com: Authentication failed.
2021-12-02T21:11:56.524 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-02T21:11:56.802 INFO:teuthology.provision.fog.smithi017:Node is ready
2021-12-02T21:11:56.820 INFO:teuthology.orchestra.run.smithi017.stdout:smithi017.front.sepia.ceph.com
2021-12-02T21:11:56.878 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17 smithi017.front.sepia.ceph.com smithi017
2021-12-02T21:11:57.231 INFO:teuthology.provision.fog.smithi017:Deploy complete!
2021-12-02T21:12:06.586 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-02T21:12:06.845 INFO:teuthology.provision.fog.smithi168:Node is ready
2021-12-02T21:12:06.902 INFO:teuthology.orchestra.run.smithi168.stdout:smithi168.front.sepia.ceph.com
2021-12-02T21:12:06.960 INFO:teuthology.orchestra.run.smithi168.stdout:172.21.15.168 smithi168.front.sepia.ceph.com smithi168
2021-12-02T21:12:07.298 INFO:teuthology.provision.fog.smithi168:Deploy complete!
2021-12-02T21:12:07.455 INFO:teuthology.lock.ops:Checking smithi017.front.sepia.ceph.com
2021-12-02T21:12:07.455 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-02T21:12:07.474 INFO:teuthology.lock.ops:Checking smithi168.front.sepia.ceph.com
2021-12-02T21:12:07.475 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-02T21:12:07.493 INFO:teuthology.lock.ops:Updating [smithi017.front.sepia.ceph.com]: set os type and version on server
2021-12-02T21:12:07.493 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com'
2021-12-02T21:12:07.736 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64
2021-12-02T21:12:07.793 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="CentOS Stream"
2021-12-02T21:12:07.793 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="8"
2021-12-02T21:12:07.794 INFO:teuthology.orchestra.run.smithi017.stdout:ID="centos"
2021-12-02T21:12:07.794 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="rhel fedora"
2021-12-02T21:12:07.794 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="8"
2021-12-02T21:12:07.794 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el8"
2021-12-02T21:12:07.794 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="CentOS Stream 8"
2021-12-02T21:12:07.795 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31"
2021-12-02T21:12:07.795 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-02T21:12:07.795 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://centos.org/"
2021-12-02T21:12:07.795 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-02T21:12:07.796 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-02T21:12:07.796 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-12-02T21:12:07.797 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server
2021-12-02T21:12:07.815 INFO:teuthology.lock.ops:Updating [smithi168.front.sepia.ceph.com]: set os type and version on server
2021-12-02T21:12:07.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi168.front.sepia.ceph.com'
2021-12-02T21:12:08.041 INFO:teuthology.orchestra.run.smithi168.stdout:x86_64
2021-12-02T21:12:08.096 INFO:teuthology.orchestra.run.smithi168.stdout:NAME="CentOS Stream"
2021-12-02T21:12:08.096 INFO:teuthology.orchestra.run.smithi168.stdout:VERSION="8"
2021-12-02T21:12:08.096 INFO:teuthology.orchestra.run.smithi168.stdout:ID="centos"
2021-12-02T21:12:08.096 INFO:teuthology.orchestra.run.smithi168.stdout:ID_LIKE="rhel fedora"
2021-12-02T21:12:08.097 INFO:teuthology.orchestra.run.smithi168.stdout:VERSION_ID="8"
2021-12-02T21:12:08.097 INFO:teuthology.orchestra.run.smithi168.stdout:PLATFORM_ID="platform:el8"
2021-12-02T21:12:08.097 INFO:teuthology.orchestra.run.smithi168.stdout:PRETTY_NAME="CentOS Stream 8"
2021-12-02T21:12:08.097 INFO:teuthology.orchestra.run.smithi168.stdout:ANSI_COLOR="0;31"
2021-12-02T21:12:08.098 INFO:teuthology.orchestra.run.smithi168.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-02T21:12:08.098 INFO:teuthology.orchestra.run.smithi168.stdout:HOME_URL="https://centos.org/"
2021-12-02T21:12:08.098 INFO:teuthology.orchestra.run.smithi168.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-02T21:12:08.098 INFO:teuthology.orchestra.run.smithi168.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-02T21:12:08.099 INFO:teuthology.orchestra.run.smithi168.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-12-02T21:12:08.099 INFO:teuthology.lock.ops:Updating smithi168.front.sepia.ceph.com on lock server
2021-12-02T21:12:08.199 INFO:teuthology.dispatcher.supervisor:Running job 6541057
2021-12-02T21:12:08.200 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology -v --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2021-12-02_20:59:09-orch:cephadm-wip-sage2-testing-2021-12-02-1025-distro-basic-smithi/6541057 --name sage-2021-12-02_20:59:09-orch:cephadm-wip-sage2-testing-2021-12-02-1025-distro-basic-smithi --description orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/sage-2021-12-02_20:59:09-orch:cephadm-wip-sage2-testing-2021-12-02-1025-distro-basic-smithi/6541057/orig.config.yaml
2021-12-02T21:12:08.203 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sage-2021-12-02_20:59:09-orch:cephadm-wip-sage2-testing-2021-12-02-1025-distro-basic-smithi/6541057
2021-12-02T21:12:08.204 INFO:teuthology.dispatcher.supervisor:Job PID: 21680
2021-12-02T21:12:08.204 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-12-03T09:12:29.282 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-12-03T09:12:29.506 INFO:teuthology.kill:Killing Pids: {21680}
2021-12-03T09:12:30.715 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2021-12-03T09:12:30.716 INFO:teuthology.task.internal:roles: ubuntu@smithi168.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2021-12-03T09:12:30.716 INFO:teuthology.misc:Compressing logs...
2021-12-03T09:12:30.717 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com'
2021-12-03T09:12:30.907 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi168.front.sepia.ceph.com'
2021-12-03T09:12:30.956 INFO:teuthology.orchestra.run.smithi017.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2021-12-03T09:12:31.141 INFO:teuthology.orchestra.run.smithi168.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2021-12-03T09:12:31.189 INFO:teuthology.orchestra.run.smithi017.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-12-03T09:12:31.189 INFO:teuthology.orchestra.run.smithi017.stderr:tar: Error is not recoverable: exiting now
2021-12-03T09:12:31.321 INFO:teuthology.orchestra.run.smithi168.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-12-03T09:12:31.322 INFO:teuthology.orchestra.run.smithi168.stderr:tar: Error is not recoverable: exiting now
2021-12-03T09:12:31.324 INFO:teuthology.misc:Compressing logs...
2021-12-03T09:12:31.910 INFO:teuthology.orchestra.run.smithi168.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-12-03T09:12:32.067 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-12-03T09:12:32.453 INFO:teuthology.misc:Compressing logs...
2021-12-03T09:12:56.125 INFO:teuthology.orchestra.run.smithi168.stderr:gzip: /var/log/ceph/fcdaffcc-53b5-11ec-8c2e-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2021-12-03T09:13:42.520 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/fcdaffcc-53b5-11ec-8c2e-001a4aab830c/ceph-mon.smithi017.log: file size changed while zipping
2021-12-03T09:13:50.166 INFO:teuthology.orchestra.run.smithi168.stderr:gzip: /var/log/ceph/fcdaffcc-53b5-11ec-8c2e-001a4aab830c/ceph-mgr.smithi168.hlcgem.log: file size changed while zipping
2021-12-03T09:14:10.894 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/fcdaffcc-53b5-11ec-8c2e-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-12-03T09:14:16.425 INFO:teuthology.orchestra.run.smithi168.stderr:gzip: /var/log/ceph/fcdaffcc-53b5-11ec-8c2e-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-12-03T09:14:28.383 INFO:teuthology.orchestra.run.smithi168.stderr:gzip: /var/log/ceph/fcdaffcc-53b5-11ec-8c2e-001a4aab830c/ceph-mon.smithi168.log: file size changed while zipping
2021-12-03T09:14:40.630 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/fcdaffcc-53b5-11ec-8c2e-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-12-03T09:14:55.334 INFO:teuthology.orchestra.run.smithi168.stderr:gzip: /var/log/ceph/fcdaffcc-53b5-11ec-8c2e-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-12-03T09:15:00.258 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/fcdaffcc-53b5-11ec-8c2e-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-12-03T09:15:27.780 INFO:teuthology.orchestra.run.smithi168.stderr:gzip: /var/log/ceph/fcdaffcc-53b5-11ec-8c2e-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2021-12-03T09:15:30.178 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/fcdaffcc-53b5-11ec-8c2e-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-12-03T09:16:40.524 INFO:teuthology.kill:No teuthology processes running
2021-12-03T09:16:41.675 INFO:teuthology.kill:Nuking machines: ['smithi017', 'smithi168']
2021-12-03T09:16:42.619 INFO:teuthology.kill:2021-12-03 09:16:42,619.619 INFO:teuthology.nuke:targets:
2021-12-03T09:16:42.620 INFO:teuthology.kill:  smithi017.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCdHoIo8VADWV8lauNz/3FT2cCed2mOQDrPGkc+R8RimRrgQTDonhdh5q+V7ScN0GFwILys2gLGIQLzoI8sf59I7tr40f/bFYOjj61WvxLgfrdM0AvstV3C3078GkbaVe2KrGs3MZgDDaJ9BQyten9j7xJjnbEHepwjqrIT+JCnOMA3BACa8RnWT4WU0CNCfWGNknLueYOqb+qiOX7nxsSHJFYpGMqAmVQ0r9wT1VDfVI6EE9JSyqynNvx4Ex5d73rrXGOaG0rbtckfdta+j5FLhe61MeE3HK8AGDchjwrCc/lhZViJhj+94Fe4OBq616fg3V146fujeyiLHpfa5XOn5S9063QKM54f/xa1Odd1aSKponISOg+uU4AY4dashWjowYWt9xsASlQRUh7WNGyXOp2pYVzwrmtLxfCmohrS1ps83nSD5F0wCYuvR+JfGqBv83MGgF0Db3jxc1ar7xA53q2XJMHz7iL0ejcmfMimV2yXNkiMFb1AJx44enLXkt0=
2021-12-03T09:16:42.621 INFO:teuthology.kill:  smithi168.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtANTUqRpzwBsOoIv0gq5MPe5lxoJji1C1rGsLlwE205Q/Kfa/+YFH0B3j3TtfcB3inZRmWImSNc1Q8n76UAZYVyQ6DqyJNustC/gxNNRtMJAdzbFRtVKY/wmq8C06xj5/OnYu6FLx+j5hxfbm6sUglk92u1ZF/9Y0ODBYLJcM+/g1/BgyULU/GwQyFMx53I3sriuxxdhI5fwqWvRIi+1Nf/Up7OXjm2FBxZ9GydP0ZmB4rfGe/QV+cP75mnWTzsN8EV9+jh/rh4/ANui4mI252MfU+ZWE8389HE9RN3BdSo+QHfx5wl9UqqlKrdeVlPgpkwKIijAUky6EsByxPLeJ/XaxZY0vCFFmacjkhAUyA543tR7YBiH08Yng9frplVg2q6zI5jURHvYztfGFx0E9c/haeADBjeLl6m36yxsfc31XyTWCp1BEV1EzwTNpFU4kfX4PGL6Z+g+9geTZtj163XSQSW/epYLci4EZf4yaUKZNmq7Gkx4eLtXFXOA1x7k=
2021-12-03T09:16:42.646 INFO:teuthology.kill:2021-12-03 09:16:42,646.646 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-03T09:16:42.648 INFO:teuthology.kill:2021-12-03 09:16:42,648.648 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-03T09:16:42.727 INFO:teuthology.kill:2021-12-03 09:16:42,726.726 INFO:teuthology.orchestra.console:Power off smithi168
2021-12-03T09:16:42.741 INFO:teuthology.kill:2021-12-03 09:16:42,741.741 INFO:teuthology.orchestra.console:Power off smithi017
2021-12-03T09:16:51.097 INFO:teuthology.kill:2021-12-03 09:16:51,096.096 INFO:teuthology.orchestra.console:Power off for smithi168 completed
2021-12-03T09:16:51.248 INFO:teuthology.kill:2021-12-03 09:16:51,248.248 INFO:teuthology.lock.ops:unlocked: smithi168.front.sepia.ceph.com
2021-12-03T09:16:55.212 INFO:teuthology.kill:2021-12-03 09:16:55,212.212 INFO:teuthology.orchestra.console:Power off for smithi017 completed
2021-12-03T09:16:55.365 INFO:teuthology.kill:2021-12-03 09:16:55,364.364 INFO:teuthology.lock.ops:unlocked: smithi017.front.sepia.ceph.com
2021-12-03T09:18:55.589 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-12-03T09:18:55.683 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi017 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-03_05:17:02-krbd-pacific-testing-default-smithi/6542191
2021-12-03T09:18:55.684 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi168 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-03_05:17:02-krbd-pacific-testing-default-smithi/6542191