2022-03-22T21:04:18.206 INFO:root:teuthology version: 1.1.0-a4e18662
2022-03-22T21:04:18.325 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi137.front.sepia.ceph.com' reimaging
2022-03-22T21:04:18.327 INFO:teuthology.lock.ops:Updating [ubuntu@smithi137.front.sepia.ceph.com]: reset os type and version on server
2022-03-22T21:04:18.327 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server
2022-03-22T21:04:18.348 INFO:teuthology.lock.ops:Node 'ubuntu@smithi137.front.sepia.ceph.com' reimaging is complete
2022-03-22T21:04:18.349 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi153.front.sepia.ceph.com' reimaging
2022-03-22T21:04:18.349 INFO:teuthology.lock.ops:Updating [ubuntu@smithi153.front.sepia.ceph.com]: reset os type and version on server
2022-03-22T21:04:18.349 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server
2022-03-22T21:04:18.369 INFO:teuthology.lock.ops:Node 'ubuntu@smithi153.front.sepia.ceph.com' reimaging is complete
2022-03-22T21:04:18.573 INFO:teuthology.provision.fog.smithi137:Scheduling deploy of centos 8.stream
2022-03-22T21:04:18.577 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of centos 8.stream
2022-03-22T21:04:18.955 INFO:teuthology.orchestra.console:Power off smithi153
2022-03-22T21:04:18.970 INFO:teuthology.orchestra.console:Power off smithi137
2022-03-22T21:04:31.441 INFO:teuthology.orchestra.console:Power off for smithi153 completed
2022-03-22T21:04:31.447 INFO:teuthology.orchestra.console:Power off for smithi137 completed
2022-03-22T21:04:31.543 INFO:teuthology.orchestra.console:Power on smithi153
2022-03-22T21:04:31.551 INFO:teuthology.orchestra.console:Power on smithi137
2022-03-22T21:04:35.806 INFO:teuthology.orchestra.console:Power on for smithi137 completed
2022-03-22T21:04:35.907 INFO:teuthology.provision.fog.smithi137:Waiting for deploy to finish
2022-03-22T21:04:39.912 INFO:teuthology.orchestra.console:Power on for smithi153 completed
2022-03-22T21:04:40.013 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish
2022-03-22T21:08:39.783 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-22T21:09:04.871 INFO:teuthology.provision.fog.smithi137:Node is ready
2022-03-22T21:09:04.894 INFO:teuthology.orchestra.run.smithi137.stdout:smithi137.front.sepia.ceph.com
2022-03-22T21:09:04.953 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137 smithi137.front.sepia.ceph.com smithi137
2022-03-22T21:09:05.326 INFO:teuthology.provision.fog.smithi137:Deploy complete!
2022-03-22T21:09:26.911 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-22T21:09:27.169 INFO:teuthology.provision.fog.smithi153:Node is ready
2022-03-22T21:09:27.188 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com
2022-03-22T21:09:27.246 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153
2022-03-22T21:09:27.583 INFO:teuthology.provision.fog.smithi153:Deploy complete!
2022-03-22T21:09:27.742 INFO:teuthology.lock.ops:Checking smithi137.front.sepia.ceph.com
2022-03-22T21:09:27.743 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-22T21:09:27.761 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com
2022-03-22T21:09:27.761 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-22T21:09:27.780 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: set os type and version on server
2022-03-22T21:09:27.780 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com'
2022-03-22T21:09:28.021 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64
2022-03-22T21:09:28.079 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream"
2022-03-22T21:09:28.080 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="8"
2022-03-22T21:09:28.080 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos"
2022-03-22T21:09:28.080 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora"
2022-03-22T21:09:28.080 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="8"
2022-03-22T21:09:28.081 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el8"
2022-03-22T21:09:28.081 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 8"
2022-03-22T21:09:28.081 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31"
2022-03-22T21:09:28.081 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-03-22T21:09:28.082 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/"
2022-03-22T21:09:28.082 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-22T21:09:28.082 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-22T21:09:28.082 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-03-22T21:09:28.084 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server
2022-03-22T21:09:28.102 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server
2022-03-22T21:09:28.102 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com'
2022-03-22T21:09:28.291 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64
2022-03-22T21:09:28.348 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream"
2022-03-22T21:09:28.349 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8"
2022-03-22T21:09:28.349 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos"
2022-03-22T21:09:28.349 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora"
2022-03-22T21:09:28.349 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8"
2022-03-22T21:09:28.350 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8"
2022-03-22T21:09:28.350 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 8"
2022-03-22T21:09:28.350 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31"
2022-03-22T21:09:28.350 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-03-22T21:09:28.351 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/"
2022-03-22T21:09:28.351 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-22T21:09:28.351 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-22T21:09:28.351 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-03-22T21:09:28.352 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server
2022-03-22T21:09:28.447 INFO:teuthology.dispatcher.supervisor:Running job 6753178
2022-03-22T21:09:28.447 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-03-22_14:10:35-rados-wip-yuri4-testing-2022-03-21-1648-pacific-distro-default-smithi/6753178 --name yuriw-2022-03-22_14:10:35-rados-wip-yuri4-testing-2022-03-21-1648-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2022-03-22_14:10:35-rados-wip-yuri4-testing-2022-03-21-1648-pacific-distro-default-smithi/6753178/orig.config.yaml
2022-03-22T21:09:28.450 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-03-22_14:10:35-rados-wip-yuri4-testing-2022-03-21-1648-pacific-distro-default-smithi/6753178
2022-03-22T21:09:28.451 INFO:teuthology.dispatcher.supervisor:Job PID: 30515
2022-03-22T21:09:28.452 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-03-23T03:39:57.099 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-03-23T03:39:57.256 INFO:teuthology.kill:Killing Pids: {30515}
2022-03-23T03:39:58.408 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-03-23T03:39:58.409 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-03-23T03:39:58.409 INFO:teuthology.misc:Compressing logs...
2022-03-23T03:39:58.410 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com'
2022-03-23T03:39:58.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com'
2022-03-23T03:39:58.659 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-03-23T03:39:58.846 INFO:teuthology.orchestra.run.smithi153.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-03-23T03:39:58.962 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-03-23T03:39:58.963 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now
2022-03-23T03:39:59.002 INFO:teuthology.orchestra.run.smithi153.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-03-23T03:39:59.002 INFO:teuthology.orchestra.run.smithi153.stderr:tar: Error is not recoverable: exiting now
2022-03-23T03:39:59.004 INFO:teuthology.misc:Compressing logs...
2022-03-23T03:39:59.259 INFO:teuthology.misc:Compressing logs...
2022-03-23T03:40:00.007 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/97e36cf0-aa25-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-03-23T03:40:00.546 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/97e36cf0-aa25-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-03-23T03:40:01.009 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/97e36cf0-aa25-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-03-23T03:40:01.565 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/97e36cf0-aa25-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-03-23T03:40:10.756 INFO:teuthology.orchestra.run.smithi137.stderr:gzip: /var/log/ceph/97e36cf0-aa25-11ec-8c35-001a4aab830c/ceph-mon.smithi137.log: file size changed while zipping
2022-03-23T03:40:30.054 INFO:teuthology.orchestra.run.smithi137.stderr:gzip: /var/log/ceph/97e36cf0-aa25-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-03-23T03:40:43.518 INFO:teuthology.orchestra.run.smithi137.stderr:gzip: /var/log/ceph/97e36cf0-aa25-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-03-23T03:40:45.931 INFO:teuthology.orchestra.run.smithi137.stderr:gzip: /var/log/ceph/97e36cf0-aa25-11ec-8c35-001a4aab830c/ceph-mgr.smithi137.vbhvau.log: file size changed while zipping
2022-03-23T03:41:09.332 INFO:teuthology.orchestra.run.smithi137.stderr:gzip: /var/log/ceph/97e36cf0-aa25-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-03-23T03:41:31.863 INFO:teuthology.orchestra.run.smithi137.stderr:gzip: /var/log/ceph/97e36cf0-aa25-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-03-23T03:42:09.753 INFO:teuthology.kill:No teuthology processes running
2022-03-23T03:42:10.944 INFO:teuthology.kill:Nuking machines: ['smithi137', 'smithi153']
2022-03-23T03:42:11.967 INFO:teuthology.kill:2022-03-23 03:42:11,967.967 INFO:teuthology.nuke:targets:
2022-03-23T03:42:11.968 INFO:teuthology.kill:  smithi137.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDY26pgh7b/oEGYSorg/aImxXnIkD30TD+04xKsZTBWruqwSNUq4EiviU8OxWtaMvPu55VqnIXIRCB0YD3waHIkokL+xD5Z66SnpwD9Ff7WpS/SKyAVGsfAG9J1vVVxkUrrtu4rj3BhDN5+34lCps5JG6q5OZcKFEp84oW3RMx/9Wd9YHjHki2TUSFxczyEuXSGLzZzRmUczgEtHWPX74nPTsIQyr1SXtFVlzI9a7Q6eAUxf6aeyzEjd42TCVWsI2I6BC2lH1W0olX99vggFZHOM5HoFaeAuKwpXtHMi8n+93UrqB5aIXA2pTg+gYDGmsRvgvAZBBfMtfite0fplfuKaQuXgyoyAz9dWaX7WpVsW+eYWcLHsBTRSgZVC/yOlyfTZ/w5ESxpJAC0wZIF6wL2yyOApVrqHf3UnZDZ+zimuzMUYLcFWFlyHqg12yFH/72Q+1p/TOESeolp6hWQzYT/3ec54fFpuFPvGeQM037Gw9YIB21WpDxfWo05o2VUHZE=
2022-03-23T03:42:11.969 INFO:teuthology.kill:  smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDsJeJPIZ+alqggC4IdjwFmalp/jJIpZV/3LASvaRsL+KXfuDo1FnqXp4vlH8TDJkEphhV84ncIsDkODtkbUHCw5oLiqgMMwwtczrZwNbKkHcPpGtj49R0JcJIgq2pt8DlYvyVFfSLwbj+qWPZjk7AFiYO1pKRfmSZl6sk/+hfbI/I88pygB6OZ9JJ+42YYzm2yOxRRh0zgXHpuXhsl5ojy7Fhf5JO+4tlw7fgWGQoYo3FoHrvoDSaPBdwVAaynZ+cokcoOhHVuc/rVwwZzQnIb25XvfoXgve0clH+Xl+ZyHYcGXaEoNCSYSbu+SUajhrXm71xwvZTmu8/4JtvmRmooDvyt5FBPv/jA84xryomsEN9hmOLS8hhmNg0mGi+789ferl/nLAxKzNh8u+QPdZxDC4lcgeo0MYDXpojA/ql8Wl6+nFQarAigU1Br62ipI7qCgYeZYOBRGZpeduF6WAjqRs4BwsfUhxB2cVWA2cG2bEb7CHMooERQFKjOHBu7/Zk=
2022-03-23T03:42:11.994 INFO:teuthology.kill:2022-03-23 03:42:11,994.994 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-23T03:42:11.997 INFO:teuthology.kill:2022-03-23 03:42:11,997.997 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-23T03:42:12.069 INFO:teuthology.kill:2022-03-23 03:42:12,069.069 INFO:teuthology.orchestra.console:Power off smithi153
2022-03-23T03:42:12.084 INFO:teuthology.kill:2022-03-23 03:42:12,084.084 INFO:teuthology.orchestra.console:Power off smithi137
2022-03-23T03:42:20.439 INFO:teuthology.kill:2022-03-23 03:42:20,438.438 INFO:teuthology.orchestra.console:Power off for smithi153 completed
2022-03-23T03:42:20.468 INFO:teuthology.kill:2022-03-23 03:42:20,468.468 INFO:teuthology.orchestra.console:Power off for smithi137 completed
2022-03-23T03:42:20.607 INFO:teuthology.kill:2022-03-23 03:42:20,605.605 INFO:teuthology.lock.ops:unlocked: smithi153.front.sepia.ceph.com
2022-03-23T03:42:20.615 INFO:teuthology.kill:2022-03-23 03:42:20,614.614 INFO:teuthology.lock.ops:unlocked: smithi137.front.sepia.ceph.com
2022-03-23T03:44:20.932 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-03-23T03:44:21.029 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi137 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-22_14:21:34-fs-wip-yuri4-testing-2022-03-21-1648-pacific-distro-default-smithi/6753366
2022-03-23T03:44:21.029 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi153 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-22_14:21:34-fs-wip-yuri4-testing-2022-03-21-1648-pacific-distro-default-smithi/6753366