2022-02-02T19:27:22.911 INFO:root:teuthology version: 1.1.0-0f634dc1
2022-02-02T19:27:23.036 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi029.front.sepia.ceph.com' reimaging
2022-02-02T19:27:23.037 INFO:teuthology.lock.ops:Updating [ubuntu@smithi029.front.sepia.ceph.com]: reset os type and version on server
2022-02-02T19:27:23.037 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server
2022-02-02T19:27:23.056 INFO:teuthology.lock.ops:Node 'ubuntu@smithi029.front.sepia.ceph.com' reimaging is complete
2022-02-02T19:27:23.057 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi169.front.sepia.ceph.com' reimaging
2022-02-02T19:27:23.057 INFO:teuthology.lock.ops:Updating [ubuntu@smithi169.front.sepia.ceph.com]: reset os type and version on server
2022-02-02T19:27:23.057 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server
2022-02-02T19:27:23.077 INFO:teuthology.lock.ops:Node 'ubuntu@smithi169.front.sepia.ceph.com' reimaging is complete
2022-02-02T19:27:23.313 INFO:teuthology.provision.fog.smithi029:Scheduling deploy of centos 8.stream
2022-02-02T19:27:23.347 INFO:teuthology.provision.fog.smithi169:Scheduling deploy of centos 8.stream
2022-02-02T19:27:23.883 INFO:teuthology.orchestra.console:Power off smithi029
2022-02-02T19:27:23.902 INFO:teuthology.orchestra.console:Power off smithi169
2022-02-02T19:27:32.256 INFO:teuthology.orchestra.console:Power off for smithi029 completed
2022-02-02T19:27:32.357 INFO:teuthology.orchestra.console:Power on smithi029
2022-02-02T19:27:36.371 INFO:teuthology.orchestra.console:Power off for smithi169 completed
2022-02-02T19:27:36.474 INFO:teuthology.orchestra.console:Power on smithi169
2022-02-02T19:27:40.724 INFO:teuthology.orchestra.console:Power on for smithi029 completed
2022-02-02T19:27:40.729 INFO:teuthology.orchestra.console:Power on for smithi169 completed
2022-02-02T19:27:40.826 INFO:teuthology.provision.fog.smithi029:Waiting for deploy to finish
2022-02-02T19:27:40.830 INFO:teuthology.provision.fog.smithi169:Waiting for deploy to finish
2022-02-02T19:32:28.461 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-02T19:32:28.797 INFO:teuthology.provision.fog.smithi029:Node is ready
2022-02-02T19:32:28.815 INFO:teuthology.orchestra.run.smithi029.stdout:smithi029.front.sepia.ceph.com
2022-02-02T19:32:28.873 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29 smithi029.front.sepia.ceph.com smithi029
2022-02-02T19:32:29.226 INFO:teuthology.provision.fog.smithi029:Deploy complete!
2022-02-02T19:32:37.406 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-02T19:32:37.706 INFO:teuthology.provision.fog.smithi169:Node is ready
2022-02-02T19:32:37.723 INFO:teuthology.orchestra.run.smithi169.stdout:smithi169.front.sepia.ceph.com
2022-02-02T19:32:37.780 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.15.169 smithi169.front.sepia.ceph.com smithi169
2022-02-02T19:32:38.109 INFO:teuthology.provision.fog.smithi169:Deploy complete!
2022-02-02T19:32:38.290 INFO:teuthology.lock.ops:Checking smithi029.front.sepia.ceph.com
2022-02-02T19:32:38.292 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-02T19:32:38.313 INFO:teuthology.lock.ops:Checking smithi169.front.sepia.ceph.com
2022-02-02T19:32:38.314 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-02T19:32:38.333 INFO:teuthology.lock.ops:Updating [smithi029.front.sepia.ceph.com]: set os type and version on server
2022-02-02T19:32:38.333 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com'
2022-02-02T19:32:38.529 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64
2022-02-02T19:32:38.588 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="CentOS Stream"
2022-02-02T19:32:38.589 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="8"
2022-02-02T19:32:38.589 INFO:teuthology.orchestra.run.smithi029.stdout:ID="centos"
2022-02-02T19:32:38.589 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="rhel fedora"
2022-02-02T19:32:38.589 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="8"
2022-02-02T19:32:38.590 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el8"
2022-02-02T19:32:38.590 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-02T19:32:38.590 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31"
2022-02-02T19:32:38.590 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-02T19:32:38.591 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://centos.org/"
2022-02-02T19:32:38.591 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-02T19:32:38.591 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-02T19:32:38.591 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-02T19:32:38.593 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server
2022-02-02T19:32:38.612 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: set os type and version on server
2022-02-02T19:32:38.612 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com'
2022-02-02T19:32:38.793 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64
2022-02-02T19:32:38.848 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream"
2022-02-02T19:32:38.848 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="8"
2022-02-02T19:32:38.848 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos"
2022-02-02T19:32:38.849 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora"
2022-02-02T19:32:38.849 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="8"
2022-02-02T19:32:38.849 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el8"
2022-02-02T19:32:38.849 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-02T19:32:38.849 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31"
2022-02-02T19:32:38.850 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-02T19:32:38.850 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/"
2022-02-02T19:32:38.850 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-02T19:32:38.850 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-02T19:32:38.851 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-02T19:32:38.851 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server
2022-02-02T19:32:38.955 INFO:teuthology.dispatcher.supervisor:Running job 6655632
2022-02-02T19:32:38.956 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-02-02_19:24:07-rados-wip-yuri5-testing-2022-02-02-0936-pacific-distro-default-smithi/6655632 --name yuriw-2022-02-02_19:24:07-rados-wip-yuri5-testing-2022-02-02-0936-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2022-02-02_19:24:07-rados-wip-yuri5-testing-2022-02-02-0936-pacific-distro-default-smithi/6655632/orig.config.yaml
2022-02-02T19:32:38.959 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-02-02_19:24:07-rados-wip-yuri5-testing-2022-02-02-0936-pacific-distro-default-smithi/6655632
2022-02-02T19:32:38.960 INFO:teuthology.dispatcher.supervisor:Job PID: 30895
2022-02-02T19:32:38.960 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-03T02:03:08.547 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-03T02:03:08.708 INFO:teuthology.kill:Killing Pids: {30895}
2022-02-03T02:03:09.909 INFO:teuthology.task.internal:roles: ubuntu@smithi029.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-02-03T02:03:09.910 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-02-03T02:03:09.911 INFO:teuthology.misc:Compressing logs...
2022-02-03T02:03:09.911 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com'
2022-02-03T02:03:10.106 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com'
2022-02-03T02:03:10.159 INFO:teuthology.orchestra.run.smithi029.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-03T02:03:10.347 INFO:teuthology.orchestra.run.smithi169.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-03T02:03:10.397 INFO:teuthology.orchestra.run.smithi029.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-03T02:03:10.397 INFO:teuthology.orchestra.run.smithi029.stderr:tar: Error is not recoverable: exiting now
2022-02-03T02:03:10.445 INFO:teuthology.orchestra.run.smithi169.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-03T02:03:10.445 INFO:teuthology.orchestra.run.smithi169.stderr:tar: Error is not recoverable: exiting now
2022-02-03T02:03:10.446 INFO:teuthology.misc:Compressing logs...
2022-02-03T02:03:10.882 INFO:teuthology.misc:Compressing logs...
2022-02-03T02:03:34.126 INFO:teuthology.orchestra.run.smithi029.stderr:gzip: /var/log/ceph/11a48bbe-8460-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-02-03T02:03:35.857 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/11a48bbe-8460-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-02-03T02:03:37.825 INFO:teuthology.orchestra.run.smithi029.stderr:gzip: /var/log/ceph/11a48bbe-8460-11ec-8c35-001a4aab830c/ceph-mgr.smithi029.guicfx.log: file size changed while zipping
2022-02-03T02:03:56.909 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/11a48bbe-8460-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-02-03T02:03:57.762 INFO:teuthology.orchestra.run.smithi029.stderr:gzip: /var/log/ceph/11a48bbe-8460-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-02-03T02:04:10.874 INFO:teuthology.orchestra.run.smithi029.stderr:gzip: /var/log/ceph/11a48bbe-8460-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-02-03T02:04:15.393 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/11a48bbe-8460-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-02-03T02:04:23.790 INFO:teuthology.orchestra.run.smithi029.stderr:gzip: /var/log/ceph/11a48bbe-8460-11ec-8c35-001a4aab830c/ceph-mon.smithi029.log: file size changed while zipping
2022-02-03T02:04:33.299 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/11a48bbe-8460-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-02-03T02:04:33.406 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/11a48bbe-8460-11ec-8c35-001a4aab830c/ceph-mgr.smithi169.puuabx.log: file size changed while zipping
2022-02-03T02:04:36.767 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/11a48bbe-8460-11ec-8c35-001a4aab830c/ceph-mon.smithi169.log: file size changed while zipping
2022-02-03T02:04:47.206 INFO:teuthology.orchestra.run.smithi029.stderr:gzip: /var/log/ceph/11a48bbe-8460-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-02-03T02:05:24.212 INFO:teuthology.kill:No teuthology processes running
2022-02-03T02:05:25.495 INFO:teuthology.kill:Nuking machines: ['smithi029', 'smithi169']
2022-02-03T02:05:26.483 INFO:teuthology.kill:2022-02-03 02:05:26,482.482 INFO:teuthology.nuke:targets:
2022-02-03T02:05:26.484 INFO:teuthology.kill:  smithi029.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUO6a+a5qZTlLLgbgfGtKxw1IcKj8YJKeobe9td+Xu8c1N8l0BuRLshJNZtn3Dg7OihZEOWf4mpvpnRHOoXdPaHBiXkhFvYfJm3D/m1KRnxY5fAKxxNHPS6DYWmQuvy3+n0VgZvUVHFQSUsHZTs/RDozOQ2I9D72VDrTEghKcco18IGJb8LsEpMODRPGnYyZ8Qhw/41iE1URMnIzwOIYp7DedYoJFlp9/wM6dvpJsHkfhTzxvq2LgETkBe8hE0/XgykaN31p/7NmpjaNp+ouwhpnEM6jCqy3xIjzI+tE8T5YaK/df1cbIK1Ezmkr8qPF6z5EcFXq5m3jA1an4srZz3NxdyV9kKJ3KsledsuwknbWjyu6A/4j5pEXr/BLcQzGntZqsoRRkwWgHb4SOhkE4fhYwic1E3CQZIAo6VMx6Vb+IYsPUXDTKvbAi/9sij/6LP4pa2UdfzxQZ+qdSMdEcGdZ591tR8bziv6nx9Pj4ISmWIAfG1ia28CbpCupWAaqk=
2022-02-03T02:05:26.484 INFO:teuthology.kill:  smithi169.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDw4OqvaPA2gb/cJSI/xcyRhFxxnTUTNGc+Txu7WQM9x0KkdDl3zOhquVeYFOT7LK37wXyFTPjuZtbgA01mxTdRZjZtHa0VNdGPbaPgA82coGSxg7yvPBIDG8lwNw5KNxRH4KwmQ3TzfJFzUh1AtcdP4y6jSSinzinYKPhEPFdr9AUtE7VHGWtMZVu7J7dhL25X7sK+I2P6q3nGIXmfzckj+REVYK2UUrahe6pgiYR4Qy6rHQ9Il3DhvYkqrrw7sz/iu53vGucap3bZIuqlTDIjbe1LhnqZNaUCf5vjJI8W+osdkVrFP6gSE0JasaAMQVfNSsodfApHeWMJugPE9372F/1n0AgBXCLUNnz6JcQTe6SneEvRZcVfxV+V2GkAmn52wwBVAPwJvW5EiXACAYX6MWNaKNgii88zDg0px9MV65DkmH4TFUJ96LhUqDpASTWIPVijfMI2qpn1YSAz6KapAHtSKB7KEr06KipVxtnVzpy9zzUs+7xpDhAsBTKoA8=
2022-02-03T02:05:26.522 INFO:teuthology.kill:2022-02-03 02:05:26,521.521 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-03T02:05:26.525 INFO:teuthology.kill:2022-02-03 02:05:26,525.525 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-03T02:05:26.616 INFO:teuthology.kill:2022-02-03 02:05:26,615.615 INFO:teuthology.orchestra.console:Power off smithi029
2022-02-03T02:05:26.630 INFO:teuthology.kill:2022-02-03 02:05:26,629.629 INFO:teuthology.orchestra.console:Power off smithi169
2022-02-03T02:05:34.990 INFO:teuthology.kill:2022-02-03 02:05:34,989.989 INFO:teuthology.orchestra.console:Power off for smithi029 completed
2022-02-03T02:05:35.136 INFO:teuthology.kill:2022-02-03 02:05:35,136.136 INFO:teuthology.lock.ops:unlocked: smithi029.front.sepia.ceph.com
2022-02-03T02:05:39.109 INFO:teuthology.kill:2022-02-03 02:05:39,108.108 INFO:teuthology.orchestra.console:Power off for smithi169 completed
2022-02-03T02:05:39.292 INFO:teuthology.kill:2022-02-03 02:05:39,292.292 INFO:teuthology.lock.ops:unlocked: smithi169.front.sepia.ceph.com
2022-02-03T02:07:39.610 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-03T02:07:39.706 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi029 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-03_01:30:32-rados-wip-yuri4-testing-2022-02-02-1557-octopus-distro-default-smithi/6656508
2022-02-03T02:07:39.707 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi169 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-03_01:30:32-rados-wip-yuri4-testing-2022-02-02-1557-octopus-distro-default-smithi/6656507