2022-01-04T06:50:17.166 INFO:root:teuthology version: 1.1.0-95a7d479
2022-01-04T06:50:17.311 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi102.front.sepia.ceph.com' reimaging
2022-01-04T06:50:17.312 INFO:teuthology.lock.ops:Updating [ubuntu@smithi102.front.sepia.ceph.com]: reset os type and version on server
2022-01-04T06:50:17.313 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server
2022-01-04T06:50:17.333 INFO:teuthology.lock.ops:Node 'ubuntu@smithi102.front.sepia.ceph.com' reimaging is complete
2022-01-04T06:50:17.333 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi154.front.sepia.ceph.com' reimaging
2022-01-04T06:50:17.334 INFO:teuthology.lock.ops:Updating [ubuntu@smithi154.front.sepia.ceph.com]: reset os type and version on server
2022-01-04T06:50:17.334 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server
2022-01-04T06:50:17.354 INFO:teuthology.lock.ops:Node 'ubuntu@smithi154.front.sepia.ceph.com' reimaging is complete
2022-01-04T06:50:17.552 INFO:teuthology.provision.fog.smithi102:Scheduling deploy of centos 8.stream
2022-01-04T06:50:17.578 INFO:teuthology.provision.fog.smithi154:Scheduling deploy of centos 8.stream
2022-01-04T06:50:17.903 INFO:teuthology.orchestra.console:Power off smithi102
2022-01-04T06:50:17.947 INFO:teuthology.orchestra.console:Power off smithi154
2022-01-04T06:50:26.307 INFO:teuthology.orchestra.console:Power off for smithi154 completed
2022-01-04T06:50:26.408 INFO:teuthology.orchestra.console:Power on smithi154
2022-01-04T06:50:30.396 INFO:teuthology.orchestra.console:Power off for smithi102 completed
2022-01-04T06:50:30.497 INFO:teuthology.orchestra.console:Power on smithi102
2022-01-04T06:50:34.757 INFO:teuthology.orchestra.console:Power on for smithi102 completed
2022-01-04T06:50:34.764 INFO:teuthology.orchestra.console:Power on for smithi154 completed
2022-01-04T06:50:34.859 INFO:teuthology.provision.fog.smithi102:Waiting for deploy to finish
2022-01-04T06:50:34.865 INFO:teuthology.provision.fog.smithi154:Waiting for deploy to finish
2022-01-04T06:54:47.086 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-04T06:55:12.605 INFO:teuthology.provision.fog.smithi154:Node is ready
2022-01-04T06:55:12.667 INFO:teuthology.orchestra.run.smithi154.stdout:smithi154.front.sepia.ceph.com
2022-01-04T06:55:12.725 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154 smithi154.front.sepia.ceph.com smithi154
2022-01-04T06:55:13.084 INFO:teuthology.provision.fog.smithi154:Deploy complete!
2022-01-04T06:55:39.708 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-04T06:55:39.981 INFO:teuthology.provision.fog.smithi102:Node is ready
2022-01-04T06:55:39.999 INFO:teuthology.orchestra.run.smithi102.stdout:smithi102.front.sepia.ceph.com
2022-01-04T06:55:40.058 INFO:teuthology.orchestra.run.smithi102.stdout:172.21.15.102 smithi102.front.sepia.ceph.com smithi102
2022-01-04T06:55:40.406 INFO:teuthology.provision.fog.smithi102:Deploy complete!
2022-01-04T06:55:40.589 INFO:teuthology.lock.ops:Checking smithi102.front.sepia.ceph.com
2022-01-04T06:55:40.590 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-04T06:55:40.612 INFO:teuthology.lock.ops:Checking smithi154.front.sepia.ceph.com
2022-01-04T06:55:40.613 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-04T06:55:40.632 INFO:teuthology.lock.ops:Updating [smithi102.front.sepia.ceph.com]: set os type and version on server
2022-01-04T06:55:40.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2022-01-04T06:55:40.827 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64
2022-01-04T06:55:40.884 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream"
2022-01-04T06:55:40.884 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="8"
2022-01-04T06:55:40.885 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos"
2022-01-04T06:55:40.885 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora"
2022-01-04T06:55:40.885 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="8"
2022-01-04T06:55:40.886 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el8"
2022-01-04T06:55:40.886 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 8"
2022-01-04T06:55:40.887 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31"
2022-01-04T06:55:40.887 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-01-04T06:55:40.887 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/"
2022-01-04T06:55:40.888 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-01-04T06:55:40.888 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-01-04T06:55:40.889 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-01-04T06:55:40.891 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server
2022-01-04T06:55:40.914 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: set os type and version on server
2022-01-04T06:55:40.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com'
2022-01-04T06:55:41.161 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64
2022-01-04T06:55:41.218 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream"
2022-01-04T06:55:41.218 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="8"
2022-01-04T06:55:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos"
2022-01-04T06:55:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora"
2022-01-04T06:55:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="8"
2022-01-04T06:55:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el8"
2022-01-04T06:55:41.220 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 8"
2022-01-04T06:55:41.220 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31"
2022-01-04T06:55:41.220 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-01-04T06:55:41.220 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/"
2022-01-04T06:55:41.221 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-01-04T06:55:41.221 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-01-04T06:55:41.221 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-01-04T06:55:41.222 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server
2022-01-04T06:55:41.372 INFO:teuthology.dispatcher.supervisor:Running job 6593862
2022-01-04T06:55:41.372 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_lflores@teuthology --archive /home/teuthworker/archive/lflores-2022-01-04_06:21:44-rados-wip-primary-balancer-distro-default-smithi/6593862 --name lflores-2022-01-04_06:21:44-rados-wip-primary-balancer-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/lflores-2022-01-04_06:21:44-rados-wip-primary-balancer-distro-default-smithi/6593862/orig.config.yaml
2022-01-04T06:55:41.376 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/lflores-2022-01-04_06:21:44-rados-wip-primary-balancer-distro-default-smithi/6593862
2022-01-04T06:55:41.377 INFO:teuthology.dispatcher.supervisor:Job PID: 26481
2022-01-04T06:55:41.378 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-01-04T18:56:01.307 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-01-04T18:56:01.479 INFO:teuthology.kill:Killing Pids: {26481}
2022-01-04T18:56:02.656 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-01-04T18:56:02.657 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-01-04T18:56:02.658 INFO:teuthology.misc:Compressing logs...
2022-01-04T18:56:02.658 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com'
2022-01-04T18:56:02.853 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com'
2022-01-04T18:56:02.902 INFO:teuthology.orchestra.run.smithi102.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-01-04T18:56:03.095 INFO:teuthology.orchestra.run.smithi154.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-01-04T18:56:03.148 INFO:teuthology.orchestra.run.smithi102.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-01-04T18:56:03.148 INFO:teuthology.orchestra.run.smithi102.stderr:tar: Error is not recoverable: exiting now
2022-01-04T18:56:03.192 INFO:teuthology.orchestra.run.smithi154.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-01-04T18:56:03.193 INFO:teuthology.orchestra.run.smithi154.stderr:tar: Error is not recoverable: exiting now
2022-01-04T18:56:03.194 INFO:teuthology.misc:Compressing logs...
2022-01-04T18:56:03.754 INFO:teuthology.misc:Compressing logs...
2022-01-04T18:56:07.998 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/0e59d0f4-6d2c-11ec-8c31-001a4aab830c/ceph-mgr.smithi154.mypztx.log: file size changed while zipping
2022-01-04T18:56:30.215 INFO:teuthology.orchestra.run.smithi102.stderr:gzip: /var/log/ceph/0e59d0f4-6d2c-11ec-8c31-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-01-04T18:56:37.209 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/0e59d0f4-6d2c-11ec-8c31-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-01-04T18:56:54.390 INFO:teuthology.orchestra.run.smithi102.stderr:gzip: /var/log/ceph/0e59d0f4-6d2c-11ec-8c31-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-01-04T18:57:01.632 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/0e59d0f4-6d2c-11ec-8c31-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-01-04T18:57:11.206 INFO:teuthology.orchestra.run.smithi102.stderr:gzip: /var/log/ceph/0e59d0f4-6d2c-11ec-8c31-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-01-04T18:57:26.485 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/0e59d0f4-6d2c-11ec-8c31-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-01-04T18:57:30.898 INFO:teuthology.orchestra.run.smithi102.stderr:gzip: /var/log/ceph/0e59d0f4-6d2c-11ec-8c31-001a4aab830c/ceph-mon.smithi102.log: file size changed while zipping
2022-01-04T18:57:48.116 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/0e59d0f4-6d2c-11ec-8c31-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-01-04T18:57:56.312 INFO:teuthology.orchestra.run.smithi102.stderr:gzip: /var/log/ceph/0e59d0f4-6d2c-11ec-8c31-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-01-04T18:57:57.937 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/0e59d0f4-6d2c-11ec-8c31-001a4aab830c/ceph-mon.smithi154.log: file size changed while zipping
2022-01-04T18:58:45.854 INFO:teuthology.kill:No teuthology processes running
2022-01-04T18:58:46.925 INFO:teuthology.kill:Nuking machines: ['smithi102', 'smithi154']
2022-01-04T18:58:47.868 INFO:teuthology.kill:2022-01-04 18:58:47,868.868 INFO:teuthology.nuke:targets:
2022-01-04T18:58:47.869 INFO:teuthology.kill:  smithi102.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4q5gLIA5AjU5paj8fEoHP6hfflAII2INyHMxjIboys5wxJV0yT9F22tYaaLrxBlQtM3yrZbMVbenNCdQ2hxrg9RtDW9GUDmOTK109/YZAzzLZWlCzA7Dfbc37eg0nt66zV+SidCYezjBXeaV1c9LN3jC3A9DjE9m459e98tVgddo5nEoGQV4LDM6V39VDG+3cJS8y5PJKkwzVTW6j9fAaByZ9X1Jm5C3Q1wTb9+Ja3+mbyqy3FFKziRxzhLoVWj4s+6avyFx66Q+60gPvpN44YVSHvXa2OaOaOriSuBxf8wPv4bIO4YWOBYy8AR5dBHDHosJPJ6lSh7LFb4lMjBpMhX+U5rwAnk9aUTZj2e+oKl91ZUjnFP4EGGJ7/MTCYbqmSeCceNMXuvF+xxNEBjr/ZAVnb15K4aRwkc5VNQSWJ2G5OkW/WsroDZz9IpLdZwhdwg7EL6CP5xKMF5qm07Vv4dUFsi7rt1bSzGOrVrERyXCGZK9D1k0DoZF8pRUqDCk=
2022-01-04T18:58:47.869 INFO:teuthology.kill:  smithi154.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjRnwdP9d7ZaDrkIvo9mnRCkbR/OQOu2UPPoz7cDA+CPw7/3i1ZTp71SMuCXph4cbZ/kP/XoN+FGIXRngU3oLE5yyZuCClUbyGktzizDMaYAX2/kvqo/HVTODtk5GvyP83JXCHTcxtoT5hBXoAXqUPb+OyRVweSVukQM42GZVQuQ55VsUrCHFi0qXuwfq/NdEbpmgTEAkBc+KSxdnkQ5oNGAVVB/tyw7wBwOfNlV78gsygnUXywCZriat1m0JRoyzk2bM8a8sjzKyu/Wu1cX1lE/pH4RARDQfZYDTR3N5kBqEESOPirDKuJ7Wn02RnrfBaChxI0sDPL8ZeaaJ6c7IMaSmHRE4JVY5XnuMwYklmC2dhrcp94pSD40fwi/p8AMt0E230g9IxYxBaiURjznMBeBJF7mgHTfOY9JTUn+iIleEyfHDaawPWHf2Z2ytEFRBD1tHcfEpP/w+2i4gSknEswlrUuo0gWNP81eUN3U/xXM+kpfpHXcS49fMNjrPZPvc=
2022-01-04T18:58:47.896 INFO:teuthology.kill:2022-01-04 18:58:47,896.896 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-04T18:58:47.897 INFO:teuthology.kill:2022-01-04 18:58:47,897.897 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-04T18:58:47.966 INFO:teuthology.kill:2022-01-04 18:58:47,965.965 INFO:teuthology.orchestra.console:Power off smithi102
2022-01-04T18:58:47.979 INFO:teuthology.kill:2022-01-04 18:58:47,978.978 INFO:teuthology.orchestra.console:Power off smithi154
2022-01-04T18:59:00.444 INFO:teuthology.kill:2022-01-04 18:59:00,444.444 INFO:teuthology.orchestra.console:Power off for smithi102 completed
2022-01-04T18:59:00.450 INFO:teuthology.kill:2022-01-04 18:59:00,449.449 INFO:teuthology.orchestra.console:Power off for smithi154 completed
2022-01-04T18:59:00.593 INFO:teuthology.kill:2022-01-04 18:59:00,592.592 INFO:teuthology.lock.ops:unlocked: smithi102.front.sepia.ceph.com
2022-01-04T18:59:00.594 INFO:teuthology.kill:2022-01-04 18:59:00,594.594 INFO:teuthology.lock.ops:unlocked: smithi154.front.sepia.ceph.com
2022-01-04T19:01:00.887 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-01-04T19:01:00.987 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi102 but it was locked by another job: /home/teuthworker/archive/mbenjamin-2022-01-04_13:41:47-rgw-wip-lc-notify-distro-basic-smithi/6594912
2022-01-04T19:01:00.987 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi154 but it was locked by another job: /home/teuthworker/archive/mbenjamin-2022-01-04_13:41:47-rgw-wip-lc-notify-distro-basic-smithi/6594912