2022-01-27T16:38:12.941 INFO:root:teuthology version: 1.1.0-7c0cb867
2022-01-27T16:38:13.061 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi082.front.sepia.ceph.com' reimaging
2022-01-27T16:38:13.062 INFO:teuthology.lock.ops:Updating [ubuntu@smithi082.front.sepia.ceph.com]: reset os type and version on server
2022-01-27T16:38:13.062 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server
2022-01-27T16:38:13.081 INFO:teuthology.lock.ops:Node 'ubuntu@smithi082.front.sepia.ceph.com' reimaging is complete
2022-01-27T16:38:13.081 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi148.front.sepia.ceph.com' reimaging
2022-01-27T16:38:13.081 INFO:teuthology.lock.ops:Updating [ubuntu@smithi148.front.sepia.ceph.com]: reset os type and version on server
2022-01-27T16:38:13.082 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server
2022-01-27T16:38:13.100 INFO:teuthology.lock.ops:Node 'ubuntu@smithi148.front.sepia.ceph.com' reimaging is complete
2022-01-27T16:38:13.295 INFO:teuthology.provision.fog.smithi082:Scheduling deploy of centos 8.stream
2022-01-27T16:38:13.317 INFO:teuthology.provision.fog.smithi148:Scheduling deploy of centos 8.stream
2022-01-27T16:38:13.651 INFO:teuthology.orchestra.console:Power off smithi082
2022-01-27T16:38:13.667 INFO:teuthology.orchestra.console:Power off smithi148
2022-01-27T16:38:22.035 INFO:teuthology.orchestra.console:Power off for smithi148 completed
2022-01-27T16:38:22.136 INFO:teuthology.orchestra.console:Power on smithi148
2022-01-27T16:38:26.141 INFO:teuthology.orchestra.console:Power off for smithi082 completed
2022-01-27T16:38:26.243 INFO:teuthology.orchestra.console:Power on smithi082
2022-01-27T16:38:30.509 INFO:teuthology.orchestra.console:Power on for smithi148 completed
2022-01-27T16:38:30.610 INFO:teuthology.provision.fog.smithi148:Waiting for deploy to finish
2022-01-27T16:38:34.615 INFO:teuthology.orchestra.console:Power on for smithi082 completed
2022-01-27T16:38:34.716 INFO:teuthology.provision.fog.smithi082:Waiting for deploy to finish
2022-01-27T16:43:15.524 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-27T16:43:15.821 INFO:teuthology.provision.fog.smithi082:Node is ready
2022-01-27T16:43:15.838 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082.front.sepia.ceph.com
2022-01-27T16:43:15.893 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82 smithi082.front.sepia.ceph.com smithi082
2022-01-27T16:43:16.220 INFO:teuthology.provision.fog.smithi082:Deploy complete!
2022-01-27T16:43:17.444 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-27T16:43:17.760 INFO:teuthology.provision.fog.smithi148:Node is ready
2022-01-27T16:43:17.777 INFO:teuthology.orchestra.run.smithi148.stdout:smithi148.front.sepia.ceph.com
2022-01-27T16:43:17.835 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.15.148 smithi148.front.sepia.ceph.com smithi148
2022-01-27T16:43:18.156 INFO:teuthology.provision.fog.smithi148:Deploy complete!
2022-01-27T16:43:18.303 INFO:teuthology.lock.ops:Checking smithi082.front.sepia.ceph.com
2022-01-27T16:43:18.304 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-27T16:43:18.323 INFO:teuthology.lock.ops:Checking smithi148.front.sepia.ceph.com
2022-01-27T16:43:18.323 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-27T16:43:18.340 INFO:teuthology.lock.ops:Updating [smithi082.front.sepia.ceph.com]: set os type and version on server
2022-01-27T16:43:18.340 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com'
2022-01-27T16:43:18.533 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64
2022-01-27T16:43:18.590 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream"
2022-01-27T16:43:18.591 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8"
2022-01-27T16:43:18.591 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos"
2022-01-27T16:43:18.591 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora"
2022-01-27T16:43:18.591 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8"
2022-01-27T16:43:18.592 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8"
2022-01-27T16:43:18.592 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 8"
2022-01-27T16:43:18.592 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31"
2022-01-27T16:43:18.592 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-01-27T16:43:18.592 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/"
2022-01-27T16:43:18.593 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-01-27T16:43:18.593 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-01-27T16:43:18.593 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-01-27T16:43:18.594 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server
2022-01-27T16:43:18.613 INFO:teuthology.lock.ops:Updating [smithi148.front.sepia.ceph.com]: set os type and version on server
2022-01-27T16:43:18.613 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com'
2022-01-27T16:43:18.798 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64
2022-01-27T16:43:18.853 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream"
2022-01-27T16:43:18.854 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="8"
2022-01-27T16:43:18.854 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos"
2022-01-27T16:43:18.854 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora"
2022-01-27T16:43:18.854 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="8"
2022-01-27T16:43:18.855 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el8"
2022-01-27T16:43:18.855 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 8"
2022-01-27T16:43:18.855 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31"
2022-01-27T16:43:18.855 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-01-27T16:43:18.855 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/"
2022-01-27T16:43:18.856 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-01-27T16:43:18.856 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-01-27T16:43:18.856 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-01-27T16:43:18.857 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server
2022-01-27T16:43:18.956 INFO:teuthology.dispatcher.supervisor:Running job 6643509
2022-01-27T16:43:18.956 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7c0cb8672986d9dbe53078a123af65593653ef7a/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-27_14:57:16-rados-wip-yuri-testing-2022-01-26-1810-pacific-distro-default-smithi/6643509 --name yuriw-2022-01-27_14:57:16-rados-wip-yuri-testing-2022-01-26-1810-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2022-01-27_14:57:16-rados-wip-yuri-testing-2022-01-26-1810-pacific-distro-default-smithi/6643509/orig.config.yaml
2022-01-27T16:43:18.960 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-27_14:57:16-rados-wip-yuri-testing-2022-01-26-1810-pacific-distro-default-smithi/6643509
2022-01-27T16:43:18.961 INFO:teuthology.dispatcher.supervisor:Job PID: 3891
2022-01-27T16:43:18.961 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-01-27T23:13:48.403 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-01-27T23:13:48.617 INFO:teuthology.kill:Killing Pids: {3891}
2022-01-27T23:13:49.745 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-01-27T23:13:49.746 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-01-27T23:13:49.746 INFO:teuthology.misc:Compressing logs...
2022-01-27T23:13:49.747 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com'
2022-01-27T23:13:49.946 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com'
2022-01-27T23:13:49.997 INFO:teuthology.orchestra.run.smithi082.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-01-27T23:13:50.182 INFO:teuthology.orchestra.run.smithi148.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-01-27T23:13:50.223 INFO:teuthology.orchestra.run.smithi082.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-01-27T23:13:50.224 INFO:teuthology.orchestra.run.smithi082.stderr:tar: Error is not recoverable: exiting now
2022-01-27T23:13:50.274 INFO:teuthology.orchestra.run.smithi148.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-01-27T23:13:50.275 INFO:teuthology.orchestra.run.smithi148.stderr:tar: Error is not recoverable: exiting now
2022-01-27T23:13:50.276 INFO:teuthology.misc:Compressing logs...
2022-01-27T23:13:50.569 INFO:teuthology.misc:Compressing logs...
2022-01-27T23:14:02.050 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/5ed573d6-7f91-11ec-8c35-001a4aab830c/ceph-mon.smithi082.log: file size changed while zipping
2022-01-27T23:14:23.963 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: /var/log/ceph/5ed573d6-7f91-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-01-27T23:14:27.295 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: /var/log/ceph/5ed573d6-7f91-11ec-8c35-001a4aab830c/ceph-mon.smithi148.log: file size changed while zipping
2022-01-27T23:14:29.430 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/5ed573d6-7f91-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-01-27T23:14:52.459 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/5ed573d6-7f91-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-01-27T23:14:54.972 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: /var/log/ceph/5ed573d6-7f91-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-01-27T23:15:09.235 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/5ed573d6-7f91-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-01-27T23:15:17.599 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: /var/log/ceph/5ed573d6-7f91-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-01-27T23:15:36.503 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/5ed573d6-7f91-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-01-27T23:15:40.090 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: /var/log/ceph/5ed573d6-7f91-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-01-27T23:15:40.972 INFO:teuthology.orchestra.run.smithi082.stderr:gzip: /var/log/ceph/5ed573d6-7f91-11ec-8c35-001a4aab830c/ceph-mgr.smithi082.pwlnmi.log: file size changed while zipping
2022-01-27T23:16:26.468 INFO:teuthology.kill:No teuthology processes running
2022-01-27T23:16:27.708 INFO:teuthology.kill:Nuking machines: ['smithi082', 'smithi148']
2022-01-27T23:16:28.718 INFO:teuthology.kill:2022-01-27 23:16:28,717.717 INFO:teuthology.nuke:targets:
2022-01-27T23:16:28.719 INFO:teuthology.kill:  smithi082.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTR6J139WcEfsx1Cc43OasED19kmAXtqG3hf1dkJ9cvGDbur9XRa58qR2ptcUQh6cMUsEnAMgW1HTaZGbQHQ3k9k2R7cathp8WpE5N4yHJioF1dG87iKY8Jf24VcG+32teA9kCUFzazbULX4R7SgXB4IRQ3mTYhaqDmt8u9IRClA0Whyt3C5izlHKgVmIcngM909RqdwZke9AVlUMe/RPZ9Fsf7kFwK0hH49eRXYb8XpxH120R9K7/Dtpi54PqVWUajMegbg5P1/jpETsMmALoY6A/Jzr8jtAIWNhz9MGej4fy9s8qdvLHjW4fT3KZU7XEGF8vrSFTV1xXv9DAFaT0FMIRGu+WQBU6jA+dunhEzewn0xe6PBw4etgDyNMaxzjo7R+VhK4xcLK+gud+PSvxjHkPsWu3AtRT/uS3DjUUmEZ8r3KR4pNE5L9r5BaS3IIUxAg7lEjoY+oEJSfum33Ay6vwCXdirz4F/0bbBTRVrRxsge9758uiVssS+WRW68E=
2022-01-27T23:16:28.719 INFO:teuthology.kill:  smithi148.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYpbzOHwS5ULFWMUlAOdIoe0IS9aL9PEl8hGhw7aPq2RJqoeNV4/f3eSltqh34/P22eL1ah7zADhUf+tR3NwKAbn1g+9dGe9huZ0VgQqAy7hc1htGtLtrmOZ0vvHq/oBlfb11/hlribaYot9IX+ZqaUZvV5B5bejnP7TkOqWXq/HbJ0WLLlQsTFE8avIxLX1kJ9ruVeaL69EmaynGb08ssZEgHZXJr5oiZwcyJ2pc9GW2tbo5JftBzw+WZ/Q5nZLAV7N8hD2DeUIU5OmPMmWn/20JE9/uboLD/i2jk9BepemMJiFdMC0i9GOyZbdLZze+ph6EvSBO7O3OmcdKAI4iRzZe25NCAH8c7y1G7PB7zlPqi5owl2A8Dr5SDxwoP5PfpOITXBJ3Y7/e1lYZYZR+OKPrjRF1qgP0QTCtg44BRUCCEdZIoooeImd9YMSVJP8WK7WjvmjLIfWy+dSWsar6Q5jzgdu2je7NwpXzYdQ/Ki4nAmoUkfa+6R6oywDQx1J0=
2022-01-27T23:16:28.744 INFO:teuthology.kill:2022-01-27 23:16:28,744.744 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-27T23:16:28.748 INFO:teuthology.kill:2022-01-27 23:16:28,748.748 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-27T23:16:28.823 INFO:teuthology.kill:2022-01-27 23:16:28,823.823 INFO:teuthology.orchestra.console:Power off smithi082
2022-01-27T23:16:28.837 INFO:teuthology.kill:2022-01-27 23:16:28,837.837 INFO:teuthology.orchestra.console:Power off smithi148
2022-01-27T23:16:37.203 INFO:teuthology.kill:2022-01-27 23:16:37,202.202 INFO:teuthology.orchestra.console:Power off for smithi148 completed
2022-01-27T23:16:37.351 INFO:teuthology.kill:2022-01-27 23:16:37,351.351 INFO:teuthology.lock.ops:unlocked: smithi148.front.sepia.ceph.com
2022-01-27T23:16:41.309 INFO:teuthology.kill:2022-01-27 23:16:41,308.308 INFO:teuthology.orchestra.console:Power off for smithi082 completed
2022-01-27T23:16:41.454 INFO:teuthology.kill:2022-01-27 23:16:41,454.454 INFO:teuthology.lock.ops:unlocked: smithi082.front.sepia.ceph.com
2022-01-27T23:18:41.698 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-01-27T23:18:41.801 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi082 but it was locked by another job: /home/teuthworker/archive/cbodley-2022-01-27_22:43:41-rgw-wip-cbodley-testing2-distro-default-smithi/6644922
2022-01-27T23:18:41.801 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi148 but it was locked by another job: /home/teuthworker/archive/cbodley-2022-01-27_22:43:41-rgw-wip-cbodley-testing2-distro-default-smithi/6644922