2021-10-29T20:19:25.807 INFO:root:teuthology version: 1.1.0-c56135d1
2021-10-29T20:19:25.931 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi017.front.sepia.ceph.com' reimaging
2021-10-29T20:19:25.932 INFO:teuthology.lock.ops:Updating [ubuntu@smithi017.front.sepia.ceph.com]: reset os type and version on server
2021-10-29T20:19:25.933 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server
2021-10-29T20:19:25.954 INFO:teuthology.lock.ops:Node 'ubuntu@smithi017.front.sepia.ceph.com' reimaging is complete
2021-10-29T20:19:25.954 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi123.front.sepia.ceph.com' reimaging
2021-10-29T20:19:25.955 INFO:teuthology.lock.ops:Updating [ubuntu@smithi123.front.sepia.ceph.com]: reset os type and version on server
2021-10-29T20:19:25.955 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server
2021-10-29T20:19:25.974 INFO:teuthology.lock.ops:Node 'ubuntu@smithi123.front.sepia.ceph.com' reimaging is complete
2021-10-29T20:19:26.168 INFO:teuthology.provision.fog.smithi017:Scheduling deploy of centos 8.2
2021-10-29T20:19:26.176 INFO:teuthology.provision.fog.smithi123:Scheduling deploy of centos 8.2
2021-10-29T20:19:26.473 INFO:teuthology.orchestra.console:Power off smithi017
2021-10-29T20:19:26.486 INFO:teuthology.orchestra.console:Power off smithi123
2021-10-29T20:19:26.615 INFO:teuthology.orchestra.console:Power off for smithi017 completed
2021-10-29T20:19:26.624 INFO:teuthology.orchestra.console:Power off for smithi123 completed
2021-10-29T20:19:26.716 INFO:teuthology.orchestra.console:Power on smithi017
2021-10-29T20:19:26.725 INFO:teuthology.orchestra.console:Power on smithi123
2021-10-29T20:19:30.969 INFO:teuthology.orchestra.console:Power on for smithi017 completed
2021-10-29T20:19:31.070 INFO:teuthology.provision.fog.smithi017:Waiting for deploy to finish
2021-10-29T20:19:35.084 INFO:teuthology.orchestra.console:Power on for smithi123 completed
2021-10-29T20:19:35.185 INFO:teuthology.provision.fog.smithi123:Waiting for deploy to finish
2021-10-29T20:24:20.536 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-29T20:24:20.836 INFO:teuthology.provision.fog.smithi017:Node is ready
2021-10-29T20:24:20.852 INFO:teuthology.orchestra.run.smithi017.stdout:smithi017.front.sepia.ceph.com
2021-10-29T20:24:20.909 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17 smithi017.front.sepia.ceph.com smithi017
2021-10-29T20:24:21.243 INFO:teuthology.provision.fog.smithi017:Deploy complete!
2021-10-29T20:24:24.588 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-29T20:24:24.883 INFO:teuthology.provision.fog.smithi123:Node is ready
2021-10-29T20:24:24.939 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123.front.sepia.ceph.com
2021-10-29T20:24:24.995 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123 smithi123.front.sepia.ceph.com smithi123
2021-10-29T20:24:25.322 INFO:teuthology.provision.fog.smithi123:Deploy complete!
2021-10-29T20:24:25.476 INFO:teuthology.lock.ops:Checking smithi017.front.sepia.ceph.com
2021-10-29T20:24:25.478 INFO:teuthology.lock.ops:New key found. Updating...
2021-10-29T20:24:25.504 INFO:teuthology.lock.ops:Checking smithi123.front.sepia.ceph.com
2021-10-29T20:24:25.504 INFO:teuthology.lock.ops:New key found. Updating...
2021-10-29T20:24:25.530 INFO:teuthology.lock.ops:Updating [smithi017.front.sepia.ceph.com]: set os type and version on server
2021-10-29T20:24:25.531 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com'
2021-10-29T20:24:25.809 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64
2021-10-29T20:24:25.865 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="CentOS Linux"
2021-10-29T20:24:25.866 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="8 (Core)"
2021-10-29T20:24:25.866 INFO:teuthology.orchestra.run.smithi017.stdout:ID="centos"
2021-10-29T20:24:25.866 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="rhel fedora"
2021-10-29T20:24:25.866 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="8"
2021-10-29T20:24:25.867 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el8"
2021-10-29T20:24:25.867 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="CentOS Linux 8 (Core)"
2021-10-29T20:24:25.867 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31"
2021-10-29T20:24:25.867 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-10-29T20:24:25.868 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://www.centos.org/"
2021-10-29T20:24:25.868 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-10-29T20:24:25.868 INFO:teuthology.orchestra.run.smithi017.stdout:
2021-10-29T20:24:25.868 INFO:teuthology.orchestra.run.smithi017.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-10-29T20:24:25.869 INFO:teuthology.orchestra.run.smithi017.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-10-29T20:24:25.869 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="centos"
2021-10-29T20:24:25.869 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8"
2021-10-29T20:24:25.869 INFO:teuthology.orchestra.run.smithi017.stdout:
2021-10-29T20:24:25.871 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server
2021-10-29T20:24:25.890 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: set os type and version on server
2021-10-29T20:24:25.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com'
2021-10-29T20:24:26.147 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64
2021-10-29T20:24:26.201 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Linux"
2021-10-29T20:24:26.202 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8 (Core)"
2021-10-29T20:24:26.202 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos"
2021-10-29T20:24:26.202 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora"
2021-10-29T20:24:26.202 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8"
2021-10-29T20:24:26.202 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8"
2021-10-29T20:24:26.203 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Linux 8 (Core)"
2021-10-29T20:24:26.203 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31"
2021-10-29T20:24:26.203 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-10-29T20:24:26.203 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.centos.org/"
2021-10-29T20:24:26.204 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-10-29T20:24:26.204 INFO:teuthology.orchestra.run.smithi123.stdout:
2021-10-29T20:24:26.204 INFO:teuthology.orchestra.run.smithi123.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-10-29T20:24:26.204 INFO:teuthology.orchestra.run.smithi123.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-10-29T20:24:26.204 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="centos"
2021-10-29T20:24:26.205 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8"
2021-10-29T20:24:26.205 INFO:teuthology.orchestra.run.smithi123.stdout:
2021-10-29T20:24:26.205 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server
2021-10-29T20:24:26.311 INFO:teuthology.dispatcher.supervisor:Running job 6467339
2021-10-29T20:24:26.312 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-10-29_17:42:41-rados-wip-yuri7-testing-2021-10-28-1307-distro-basic-smithi/6467339 --name yuriw-2021-10-29_17:42:41-rados-wip-yuri7-testing-2021-10-28-1307-distro-basic-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-centos_8.2_container_tools_3.0 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2021-10-29_17:42:41-rados-wip-yuri7-testing-2021-10-28-1307-distro-basic-smithi/6467339/orig.config.yaml
2021-10-29T20:24:26.315 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-10-29_17:42:41-rados-wip-yuri7-testing-2021-10-28-1307-distro-basic-smithi/6467339
2021-10-29T20:24:26.316 INFO:teuthology.dispatcher.supervisor:Job PID: 15575
2021-10-29T20:24:26.316 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-10-30T08:25:21.007 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-10-30T08:25:21.199 INFO:teuthology.kill:Killing Pids: {15575}
2021-10-30T08:25:22.328 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2021-10-30T08:25:22.329 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2021-10-30T08:25:22.329 INFO:teuthology.misc:Compressing logs...
2021-10-30T08:25:22.330 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com'
2021-10-30T08:25:22.541 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com'
2021-10-30T08:25:22.588 INFO:teuthology.orchestra.run.smithi017.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2021-10-30T08:25:22.833 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2021-10-30T08:25:22.878 INFO:teuthology.orchestra.run.smithi017.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-10-30T08:25:22.878 INFO:teuthology.orchestra.run.smithi017.stderr:tar: Error is not recoverable: exiting now
2021-10-30T08:25:22.912 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-10-30T08:25:22.912 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now
2021-10-30T08:25:22.913 INFO:teuthology.misc:Compressing logs...
2021-10-30T08:25:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-10-30T08:25:23.537 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-10-30T08:25:23.890 INFO:teuthology.misc:Compressing logs...
2021-10-30T08:25:31.702 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/88d3c57e-38f7-11ec-8c28-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-10-30T08:25:32.357 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/88d3c57e-38f7-11ec-8c28-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-10-30T08:25:41.589 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/88d3c57e-38f7-11ec-8c28-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-10-30T08:25:42.429 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/88d3c57e-38f7-11ec-8c28-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-10-30T08:25:49.168 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/88d3c57e-38f7-11ec-8c28-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2021-10-30T08:25:58.368 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/88d3c57e-38f7-11ec-8c28-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2021-10-30T08:25:58.709 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/88d3c57e-38f7-11ec-8c28-001a4aab830c/ceph-mon.smithi017.log: file size changed while zipping
2021-10-30T08:26:01.129 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/88d3c57e-38f7-11ec-8c28-001a4aab830c/ceph-mon.smithi123.log: file size changed while zipping
2021-10-30T08:26:03.494 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/88d3c57e-38f7-11ec-8c28-001a4aab830c/ceph-mgr.smithi017.mekvru.log: file size changed while zipping
2021-10-30T08:26:13.795 INFO:teuthology.orchestra.run.smithi017.stderr:gzip: /var/log/ceph/88d3c57e-38f7-11ec-8c28-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-10-30T08:27:08.740 INFO:teuthology.orchestra.run.smithi123.stderr:tar: .: file changed as we read it
2021-10-30T08:27:08.811 INFO:teuthology.kill:No teuthology processes running
2021-10-30T08:27:09.954 INFO:teuthology.kill:Nuking machines: ['smithi017', 'smithi123']
2021-10-30T08:27:10.975 INFO:teuthology.kill:2021-10-30 08:27:10,975.975 INFO:teuthology.nuke:targets:
2021-10-30T08:27:10.976 INFO:teuthology.kill:  smithi017.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+Yoee86K9EVf9P2b6GMYuIRp9ROte/V3ULV7S1hsb92bRiAmHcJlPXwhasB6KUWZ0u0VEG5Q5mZ80iy0vvhP76VCFQRIqgdAOJaHh/44efqmKGTgy3zLthUnnwfPffTzy3uSASn35EpzK36hg6uPHfmhgZi7VuBn0HR4IC1CQvz+69rbFJG6qSZN+IUMnDWkG0KRLG1841QF4WePMddGjFDbf12TIu9VOr6SUEzzZjoxT33lExLSa6UQvWJphfc8S0h2j+X6ptT9c49Jfp0efvOew/5pBy8kp/0Xnx1wfZ+zg3vGLLs9kAWlt+8Vulnik0/5mKlE1bPJdNGSSVo6d8o+B+5GSXVVyP5yIQC6eryAra0QiF8Ueu7UVOCC0Dx1Ad/3GH6urTvogDdAMc3JhnJJHBCSqR3BdY+MYtyycTFeLGC57JNf6VAcx5tj8s10cKozGeu2db3B6gn4Mew/0eGSzghuNbPvjIbjVzzBIlGfXIYFuqinvUajNmNyHQ3M=
2021-10-30T08:27:10.977 INFO:teuthology.kill:  smithi123.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/HVqIqAIHmfcQ92n9+DnTL3QJGPuHh6SrzphFwD6xpUCFnWFqwUDFFNWEH6wAzRXtehh8YGflIA3xBvYzienbrLAKj9xMobi9je2TQZuVO+xM3PP5OlmcdXhKOsiO7FMQtoVhoCV1+/9T+mgCYTtr+P2fguZTGHiDSgy7/jOL7ZYbY217xYXfqT4jBOZ5jJM/XNK3+xhxjiE3Si80pVlhlU8CfHscfLzvNnkaGkJ//MtdlwCFP+KavoXGiot2Rdiuc0hwBZTdinHt4/yGk4zLzTksU9TgEQr0a5ym4EFERyh+PTfnEhmb4nAmk41VcyL5WngVlKr8NBOMEbZ3uc72CB90qDqFqWBkdNCQd3pSAtu00R6ODS+k3qvMp4ah0xEP/5u1KL2ME/YbKUzZXgyg+povUjPd3kYrWZCux4C4vVL3z81nf5oNDAIxwW5O+EGS/uyNsT/z1v+SMBmdKBkMxIbZQLwx9jeiJfUnt3APn0zk2f5NfhaDpG+FCy8kvO0=
2021-10-30T08:27:11.006 INFO:teuthology.kill:2021-10-30 08:27:11,006.006 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-30T08:27:11.011 INFO:teuthology.kill:2021-10-30 08:27:11,010.010 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-30T08:27:11.092 INFO:teuthology.kill:2021-10-30 08:27:11,092.092 INFO:teuthology.orchestra.console:Power off smithi123
2021-10-30T08:27:11.106 INFO:teuthology.kill:2021-10-30 08:27:11,106.106 INFO:teuthology.orchestra.console:Power off smithi017
2021-10-30T08:27:19.462 INFO:teuthology.kill:2021-10-30 08:27:19,461.461 INFO:teuthology.orchestra.console:Power off for smithi123 completed
2021-10-30T08:27:19.619 INFO:teuthology.kill:2021-10-30 08:27:19,618.618 INFO:teuthology.lock.ops:unlocked: smithi123.front.sepia.ceph.com
2021-10-30T08:27:23.584 INFO:teuthology.kill:2021-10-30 08:27:23,583.583 INFO:teuthology.orchestra.console:Power off for smithi017 completed
2021-10-30T08:27:23.729 INFO:teuthology.kill:2021-10-30 08:27:23,729.729 INFO:teuthology.lock.ops:unlocked: smithi017.front.sepia.ceph.com
2021-10-30T08:29:24.051 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-10-30T08:29:24.149 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi017 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-10-29_23:44:51-rados-wip-yuri-master-10.29.21-distro-basic-smithi/6468349
2021-10-30T08:29:24.149 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi123 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-10-29_23:44:51-rados-wip-yuri-master-10.29.21-distro-basic-smithi/6468349