2022-02-17T17:13:23.568 INFO:root:teuthology version: 1.1.0-c6442aa9
2022-02-17T17:13:23.736 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging
2022-02-17T17:13:23.737 INFO:teuthology.lock.ops:Updating [ubuntu@smithi146.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T17:13:23.738 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2022-02-17T17:13:23.759 INFO:teuthology.lock.ops:Node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging is complete
2022-02-17T17:13:23.760 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging
2022-02-17T17:13:23.761 INFO:teuthology.lock.ops:Updating [ubuntu@smithi170.front.sepia.ceph.com]: reset os type and version on server
2022-02-17T17:13:23.761 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server
2022-02-17T17:13:23.785 INFO:teuthology.lock.ops:Node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging is complete
2022-02-17T17:13:23.979 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of centos 8.stream
2022-02-17T17:13:23.985 INFO:teuthology.provision.fog.smithi170:Scheduling deploy of centos 8.stream
2022-02-17T17:13:24.300 INFO:teuthology.orchestra.console:Power off smithi146
2022-02-17T17:13:24.364 INFO:teuthology.orchestra.console:Power off smithi170
2022-02-17T17:13:32.729 INFO:teuthology.orchestra.console:Power off for smithi170 completed
2022-02-17T17:13:32.830 INFO:teuthology.orchestra.console:Power on smithi170
2022-02-17T17:13:36.783 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2022-02-17T17:13:36.885 INFO:teuthology.orchestra.console:Power on smithi146
2022-02-17T17:13:41.195 INFO:teuthology.orchestra.console:Power on for smithi170 completed
2022-02-17T17:13:41.296 INFO:teuthology.provision.fog.smithi170:Waiting for deploy to finish
2022-02-17T17:13:45.262 INFO:teuthology.orchestra.console:Power on for smithi146 completed
2022-02-17T17:13:45.363 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish
2022-02-17T17:18:13.206 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T17:18:13.453 INFO:teuthology.provision.fog.smithi170:Node is ready
2022-02-17T17:18:13.510 INFO:teuthology.orchestra.run.smithi170.stdout:smithi170.front.sepia.ceph.com
2022-02-17T17:18:13.568 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170 smithi170.front.sepia.ceph.com smithi170
2022-02-17T17:18:13.927 INFO:teuthology.provision.fog.smithi170:Deploy complete!
2022-02-17T17:18:26.197 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-17T17:18:26.437 INFO:teuthology.provision.fog.smithi146:Node is ready
2022-02-17T17:18:26.455 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com
2022-02-17T17:18:26.512 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146
2022-02-17T17:18:26.854 INFO:teuthology.provision.fog.smithi146:Deploy complete!
2022-02-17T17:18:27.003 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com
2022-02-17T17:18:27.004 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T17:18:27.026 INFO:teuthology.lock.ops:Checking smithi170.front.sepia.ceph.com
2022-02-17T17:18:27.026 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-17T17:18:27.047 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server
2022-02-17T17:18:27.047 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2022-02-17T17:18:27.239 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64
2022-02-17T17:18:27.296 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream"
2022-02-17T17:18:27.296 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8"
2022-02-17T17:18:27.296 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos"
2022-02-17T17:18:27.296 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora"
2022-02-17T17:18:27.297 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8"
2022-02-17T17:18:27.297 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8"
2022-02-17T17:18:27.297 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-17T17:18:27.297 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31"
2022-02-17T17:18:27.298 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-17T17:18:27.298 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/"
2022-02-17T17:18:27.298 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-17T17:18:27.298 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-17T17:18:27.299 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-17T17:18:27.300 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2022-02-17T17:18:27.320 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: set os type and version on server
2022-02-17T17:18:27.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com'
2022-02-17T17:18:27.566 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64
2022-02-17T17:18:27.625 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream"
2022-02-17T17:18:27.625 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8"
2022-02-17T17:18:27.625 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos"
2022-02-17T17:18:27.626 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora"
2022-02-17T17:18:27.626 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8"
2022-02-17T17:18:27.626 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8"
2022-02-17T17:18:27.626 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-17T17:18:27.627 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31"
2022-02-17T17:18:27.627 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-17T17:18:27.627 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/"
2022-02-17T17:18:27.627 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-17T17:18:27.628 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-17T17:18:27.628 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-17T17:18:27.629 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server
2022-02-17T17:18:27.730 INFO:teuthology.dispatcher.supervisor:Running job 6691442
2022-02-17T17:18:27.731 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c6442aa9e509aace1201390ea4c2d6a80e3d0800/virtualenv/bin/teuthology -v --owner scheduled_nojha@teuthology --archive /home/teuthworker/archive/nojha-2022-02-17_16:31:20-rados-wip-mgr-fixes-pacific-gil-distro-basic-smithi/6691442 --name nojha-2022-02-17_16:31:20-rados-wip-mgr-fixes-pacific-gil-distro-basic-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/nojha-2022-02-17_16:31:20-rados-wip-mgr-fixes-pacific-gil-distro-basic-smithi/6691442/orig.config.yaml
2022-02-17T17:18:27.736 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nojha-2022-02-17_16:31:20-rados-wip-mgr-fixes-pacific-gil-distro-basic-smithi/6691442
2022-02-17T17:18:27.737 INFO:teuthology.dispatcher.supervisor:Job PID: 15558
2022-02-17T17:18:27.738 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-17T23:48:55.267 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-17T23:48:55.475 INFO:teuthology.kill:Killing Pids: {15558}
2022-02-17T23:48:56.689 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-02-17T23:48:56.690 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-02-17T23:48:56.690 INFO:teuthology.misc:Compressing logs...
2022-02-17T23:48:56.691 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2022-02-17T23:48:56.884 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com'
2022-02-17T23:48:56.934 INFO:teuthology.orchestra.run.smithi146.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-17T23:48:57.122 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-17T23:48:57.169 INFO:teuthology.orchestra.run.smithi146.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-17T23:48:57.170 INFO:teuthology.orchestra.run.smithi146.stderr:tar: Error is not recoverable: exiting now
2022-02-17T23:48:57.211 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-17T23:48:57.212 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now
2022-02-17T23:48:57.213 INFO:teuthology.misc:Compressing logs...
2022-02-17T23:48:57.617 INFO:teuthology.misc:Compressing logs...
2022-02-17T23:49:19.574 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/b8fded26-9016-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-02-17T23:49:22.628 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/b8fded26-9016-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-02-17T23:49:35.366 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/b8fded26-9016-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-02-17T23:49:38.178 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/b8fded26-9016-11ec-8c35-001a4aab830c/ceph-mgr.smithi146.jgdsag.log: file size changed while zipping
2022-02-17T23:49:50.418 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/b8fded26-9016-11ec-8c35-001a4aab830c/ceph-mon.smithi146.log: file size changed while zipping
2022-02-17T23:49:53.901 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/b8fded26-9016-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-02-17T23:50:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/b8fded26-9016-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-02-17T23:50:15.759 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/b8fded26-9016-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-02-17T23:50:18.421 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/b8fded26-9016-11ec-8c35-001a4aab830c/ceph-mon.smithi170.log: file size changed while zipping
2022-02-17T23:50:39.999 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/b8fded26-9016-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-02-17T23:50:40.883 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/b8fded26-9016-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-02-17T23:51:02.343 INFO:teuthology.orchestra.run.smithi146.stderr:tar: .: file changed as we read it
2022-02-17T23:51:22.414 INFO:teuthology.kill:No teuthology processes running
2022-02-17T23:51:23.451 INFO:teuthology.kill:Nuking machines: ['smithi146', 'smithi170']
2022-02-17T23:51:24.358 INFO:teuthology.kill:2022-02-17 23:51:24,357.357 INFO:teuthology.nuke:targets:
2022-02-17T23:51:24.359 INFO:teuthology.kill:  smithi146.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWuSVDOS6+0T6yrhJd9J00jRlV2NEA+tcKLbDEQbbZvcUnwE7Yj0R+tqn4hyGqP64+0dhRaiRqb7Kk07rBwBJoKRKbKlna1J49FsNVE2G4DaDP+JyPKfnyIs8k8QDptof8V9kkSUsTFrxUI9FooIV6vuQ6tZnbE+PtO2QUDvaNdIn/i4+NYNKs41YHUcwdl1L3PJpmCl1ytE9NUzrRv9ulQLq8rEaMzlNLmzm6kYF97oG4zvORrwIYtzhJhnd937CJeTHdxiGUXf/gC5QAjO9Tdn1QxyZDdy/glAAuI/XRP1jKmE8yJSmSQQ9eLMD1nFyAepJuyU8TQMrXysytDYR6Jv59oBexIaszhyZ6altOVn52QZ29ASemdvuc9vQz/PPwoupYSnkx/UEEX+z+UWuDd+SDA9chZZA3kuGeLLkmL+VP53vn1Kvz1Pf/zSaXivkPrC/q3stbi4oO7b+RpJBIx8nh9MhvzdRoYsEJExURscBLZ7SFxVJR2UgiPS1OH90=
2022-02-17T23:51:24.359 INFO:teuthology.kill:  smithi170.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDoRNwFmhxvRQlvvJFQzri9IBUdDjcgi1+S1CnOj399FlRlJdHASbr/VQLVUTsZqg8Pn0vY9DyszUA5GGv2DvANPAMH798ts6qUqLFXJfa9iu63uadZJrG5gOuIvlnc/HMQeLFQcUTqFVr/PLmtKKI/8m5gVbAwAbU9KaTYMrtQabTX3GpM/d7VkAn11Y0SYo3dTrjpQZlvZi9axDRbKvzLc6ZanKvwPX1BQVQ9GmW9ePl2ne+4Dwb3bvrFUEO5sCKg4tP6UuprVBT8XiqF0hzLbPudLTY9ljpmD3Ot6dSeHVwNH4YejTgO4cYJJufmpy/yljNYklJhleEbXpVLPXKHbSs7qVLB0uEHMkoy28c5/smWdu5Jr13KNtNXKXBd+drqGOXDXFtj4RL7GYTgonRqH3wNpDxywY/gp17l9jLrG8d9ma99sQ9WbAnIptxDQJAUgpJI3ZuiDO+z8bOPIQI7k9suvqpekv1O6tbq59Vc/mYp+YI0lihELpGY9RTpPH8=
2022-02-17T23:51:24.384 INFO:teuthology.kill:2022-02-17 23:51:24,383.383 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T23:51:24.387 INFO:teuthology.kill:2022-02-17 23:51:24,387.387 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-17T23:51:24.465 INFO:teuthology.kill:2022-02-17 23:51:24,465.465 INFO:teuthology.orchestra.console:Power off smithi146
2022-02-17T23:51:24.478 INFO:teuthology.kill:2022-02-17 23:51:24,477.477 INFO:teuthology.orchestra.console:Power off smithi170
2022-02-17T23:51:32.841 INFO:teuthology.kill:2022-02-17 23:51:32,841.841 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2022-02-17T23:51:32.985 INFO:teuthology.kill:2022-02-17 23:51:32,985.985 INFO:teuthology.lock.ops:unlocked: smithi146.front.sepia.ceph.com
2022-02-17T23:51:36.959 INFO:teuthology.kill:2022-02-17 23:51:36,959.959 INFO:teuthology.orchestra.console:Power off for smithi170 completed
2022-02-17T23:51:37.113 INFO:teuthology.kill:2022-02-17 23:51:37,113.113 INFO:teuthology.lock.ops:unlocked: smithi170.front.sepia.ceph.com
2022-02-17T23:53:37.411 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-17T23:53:37.513 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi146 but it was locked by another job: /home/teuthworker/archive/dis-2022-02-16_22:25:18-rbd-wip-rbd-quincy-batch-3-distro-default-smithi/6689858
2022-02-17T23:53:37.513 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi170 but it was locked by another job: /home/teuthworker/archive/dis-2022-02-16_22:25:18-rbd-wip-rbd-quincy-batch-3-distro-default-smithi/6689858