2022-04-06T23:58:58.587 INFO:root:teuthology version: 1.1.0-3dd79763
2022-04-06T23:58:58.710 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi166.front.sepia.ceph.com' reimaging
2022-04-06T23:58:58.711 INFO:teuthology.lock.ops:Updating [ubuntu@smithi166.front.sepia.ceph.com]: reset os type and version on server
2022-04-06T23:58:58.712 INFO:teuthology.lock.ops:Updating smithi166.front.sepia.ceph.com on lock server
2022-04-06T23:58:58.732 INFO:teuthology.lock.ops:Node 'ubuntu@smithi166.front.sepia.ceph.com' reimaging is complete
2022-04-06T23:58:58.733 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi189.front.sepia.ceph.com' reimaging
2022-04-06T23:58:58.733 INFO:teuthology.lock.ops:Updating [ubuntu@smithi189.front.sepia.ceph.com]: reset os type and version on server
2022-04-06T23:58:58.734 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server
2022-04-06T23:58:58.755 INFO:teuthology.lock.ops:Node 'ubuntu@smithi189.front.sepia.ceph.com' reimaging is complete
2022-04-06T23:58:58.942 INFO:teuthology.provision.fog.smithi166:Scheduling deploy of centos 8.stream
2022-04-06T23:58:58.973 INFO:teuthology.provision.fog.smithi189:Scheduling deploy of centos 8.stream
2022-04-06T23:58:59.288 INFO:teuthology.orchestra.console:Power off smithi166
2022-04-06T23:58:59.303 INFO:teuthology.orchestra.console:Power off smithi189
2022-04-06T23:59:07.665 INFO:teuthology.orchestra.console:Power off for smithi166 completed
2022-04-06T23:59:07.766 INFO:teuthology.orchestra.console:Power on smithi166
2022-04-06T23:59:11.790 INFO:teuthology.orchestra.console:Power off for smithi189 completed
2022-04-06T23:59:11.892 INFO:teuthology.orchestra.console:Power on smithi189
2022-04-06T23:59:16.140 INFO:teuthology.orchestra.console:Power on for smithi166 completed
2022-04-06T23:59:16.145 INFO:teuthology.orchestra.console:Power on for smithi189 completed
2022-04-06T23:59:16.242 INFO:teuthology.provision.fog.smithi166:Waiting for deploy to finish
2022-04-06T23:59:16.246 INFO:teuthology.provision.fog.smithi189:Waiting for deploy to finish
2022-04-07T00:03:43.974 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-07T00:03:54.292 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-04-07T00:03:54.569 INFO:teuthology.provision.fog.smithi166:Node is ready
2022-04-07T00:03:54.587 INFO:teuthology.orchestra.run.smithi166.stdout:smithi166.front.sepia.ceph.com
2022-04-07T00:03:54.646 INFO:teuthology.orchestra.run.smithi166.stdout:172.21.15.166 smithi166.front.sepia.ceph.com smithi166
2022-04-07T00:03:54.983 INFO:teuthology.provision.fog.smithi166:Deploy complete!
2022-04-07T00:04:08.894 INFO:teuthology.provision.fog.smithi189:Node is ready
2022-04-07T00:04:08.912 INFO:teuthology.orchestra.run.smithi189.stdout:smithi189.front.sepia.ceph.com
2022-04-07T00:04:08.968 INFO:teuthology.orchestra.run.smithi189.stdout:172.21.15.189 smithi189.front.sepia.ceph.com smithi189
2022-04-07T00:04:09.322 INFO:teuthology.provision.fog.smithi189:Deploy complete!
2022-04-07T00:04:09.502 INFO:teuthology.lock.ops:Checking smithi166.front.sepia.ceph.com
2022-04-07T00:04:09.503 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-07T00:04:09.521 INFO:teuthology.lock.ops:Checking smithi189.front.sepia.ceph.com
2022-04-07T00:04:09.522 INFO:teuthology.lock.ops:New key found. Updating...
2022-04-07T00:04:09.540 INFO:teuthology.lock.ops:Updating [smithi166.front.sepia.ceph.com]: set os type and version on server
2022-04-07T00:04:09.541 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi166.front.sepia.ceph.com'
2022-04-07T00:04:09.784 INFO:teuthology.orchestra.run.smithi166.stdout:x86_64
2022-04-07T00:04:09.841 INFO:teuthology.orchestra.run.smithi166.stdout:NAME="CentOS Stream"
2022-04-07T00:04:09.842 INFO:teuthology.orchestra.run.smithi166.stdout:VERSION="8"
2022-04-07T00:04:09.842 INFO:teuthology.orchestra.run.smithi166.stdout:ID="centos"
2022-04-07T00:04:09.843 INFO:teuthology.orchestra.run.smithi166.stdout:ID_LIKE="rhel fedora"
2022-04-07T00:04:09.843 INFO:teuthology.orchestra.run.smithi166.stdout:VERSION_ID="8"
2022-04-07T00:04:09.843 INFO:teuthology.orchestra.run.smithi166.stdout:PLATFORM_ID="platform:el8"
2022-04-07T00:04:09.843 INFO:teuthology.orchestra.run.smithi166.stdout:PRETTY_NAME="CentOS Stream 8"
2022-04-07T00:04:09.844 INFO:teuthology.orchestra.run.smithi166.stdout:ANSI_COLOR="0;31"
2022-04-07T00:04:09.844 INFO:teuthology.orchestra.run.smithi166.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-04-07T00:04:09.844 INFO:teuthology.orchestra.run.smithi166.stdout:HOME_URL="https://centos.org/"
2022-04-07T00:04:09.844 INFO:teuthology.orchestra.run.smithi166.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-07T00:04:09.844 INFO:teuthology.orchestra.run.smithi166.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-07T00:04:09.845 INFO:teuthology.orchestra.run.smithi166.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-04-07T00:04:09.846 INFO:teuthology.lock.ops:Updating smithi166.front.sepia.ceph.com on lock server
2022-04-07T00:04:09.864 INFO:teuthology.lock.ops:Updating [smithi189.front.sepia.ceph.com]: set os type and version on server
2022-04-07T00:04:09.865 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com'
2022-04-07T00:04:10.061 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64
2022-04-07T00:04:10.118 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="CentOS Stream"
2022-04-07T00:04:10.119 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="8"
2022-04-07T00:04:10.119 INFO:teuthology.orchestra.run.smithi189.stdout:ID="centos"
2022-04-07T00:04:10.119 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="rhel fedora"
2022-04-07T00:04:10.120 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="8"
2022-04-07T00:04:10.120 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el8"
2022-04-07T00:04:10.120 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="CentOS Stream 8"
2022-04-07T00:04:10.121 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31"
2022-04-07T00:04:10.121 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-04-07T00:04:10.121 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://centos.org/"
2022-04-07T00:04:10.121 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-04-07T00:04:10.122 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-04-07T00:04:10.122 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-04-07T00:04:10.123 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server
2022-04-07T00:04:10.228 INFO:teuthology.dispatcher.supervisor:Running job 6779766
2022-04-07T00:04:10.229 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3dd797633f37e2d7e2c4ae3ef9ac35a188c2028d/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-04-06_14:21:25-orch-wip-yuri2-testing-2022-03-31-1523-pacific-distro-default-smithi/6779766 --name yuriw-2022-04-06_14:21:25-orch-wip-yuri2-testing-2022-03-31-1523-pacific-distro-default-smithi --description orch/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2022-04-06_14:21:25-orch-wip-yuri2-testing-2022-03-31-1523-pacific-distro-default-smithi/6779766/orig.config.yaml
2022-04-07T00:04:10.233 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-04-06_14:21:25-orch-wip-yuri2-testing-2022-03-31-1523-pacific-distro-default-smithi/6779766
2022-04-07T00:04:10.233 INFO:teuthology.dispatcher.supervisor:Job PID: 25145
2022-04-07T00:04:10.234 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-04-07T06:34:39.411 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-04-07T06:34:39.607 INFO:teuthology.kill:Killing Pids: {25145}
2022-04-07T06:34:40.862 INFO:teuthology.task.internal:roles: ubuntu@smithi166.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-04-07T06:34:40.863 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-04-07T06:34:40.863 INFO:teuthology.misc:Compressing logs...
2022-04-07T06:34:40.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi166.front.sepia.ceph.com'
2022-04-07T06:34:41.049 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com'
2022-04-07T06:34:41.099 INFO:teuthology.orchestra.run.smithi166.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-04-07T06:34:41.277 INFO:teuthology.orchestra.run.smithi189.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-04-07T06:34:41.319 INFO:teuthology.orchestra.run.smithi166.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-04-07T06:34:41.319 INFO:teuthology.orchestra.run.smithi166.stderr:tar: Error is not recoverable: exiting now
2022-04-07T06:34:41.366 INFO:teuthology.orchestra.run.smithi189.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-04-07T06:34:41.367 INFO:teuthology.orchestra.run.smithi189.stderr:tar: Error is not recoverable: exiting now
2022-04-07T06:34:41.368 INFO:teuthology.misc:Compressing logs...
2022-04-07T06:34:41.663 INFO:teuthology.misc:Compressing logs...
2022-04-07T06:34:59.599 INFO:teuthology.orchestra.run.smithi189.stderr:gzip: /var/log/ceph/e22be7a0-b606-11ec-8c36-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-04-07T06:34:59.931 INFO:teuthology.orchestra.run.smithi166.stderr:gzip: /var/log/ceph/e22be7a0-b606-11ec-8c36-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-04-07T06:35:11.390 INFO:teuthology.orchestra.run.smithi166.stderr:gzip: /var/log/ceph/e22be7a0-b606-11ec-8c36-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-04-07T06:35:20.951 INFO:teuthology.orchestra.run.smithi189.stderr:gzip: /var/log/ceph/e22be7a0-b606-11ec-8c36-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-04-07T06:35:21.209 INFO:teuthology.orchestra.run.smithi166.stderr:gzip: /var/log/ceph/e22be7a0-b606-11ec-8c36-001a4aab830c/ceph-mon.smithi166.log: file size changed while zipping
2022-04-07T06:35:23.658 INFO:teuthology.orchestra.run.smithi189.stderr:gzip: /var/log/ceph/e22be7a0-b606-11ec-8c36-001a4aab830c/ceph-mon.smithi189.log: file size changed while zipping
2022-04-07T06:35:39.885 INFO:teuthology.orchestra.run.smithi166.stderr:gzip: /var/log/ceph/e22be7a0-b606-11ec-8c36-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-04-07T06:35:40.466 INFO:teuthology.orchestra.run.smithi189.stderr:gzip: /var/log/ceph/e22be7a0-b606-11ec-8c36-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-04-07T06:35:55.511 INFO:teuthology.orchestra.run.smithi189.stderr:gzip: /var/log/ceph/e22be7a0-b606-11ec-8c36-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-04-07T06:35:55.749 INFO:teuthology.orchestra.run.smithi166.stderr:gzip: /var/log/ceph/e22be7a0-b606-11ec-8c36-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-04-07T06:35:57.115 INFO:teuthology.orchestra.run.smithi189.stderr:gzip: /var/log/ceph/e22be7a0-b606-11ec-8c36-001a4aab830c/ceph-mgr.smithi189.sdaduf.log: file size changed while zipping
2022-04-07T06:36:27.913 INFO:teuthology.kill:No teuthology processes running
2022-04-07T06:36:29.057 INFO:teuthology.kill:Nuking machines: ['smithi166', 'smithi189']
2022-04-07T06:36:30.045 INFO:teuthology.kill:2022-04-07 06:36:30,044.044 INFO:teuthology.nuke:targets:
2022-04-07T06:36:30.046 INFO:teuthology.kill:  smithi166.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCz//5Y6V4Ipw9vRgkPCoYNve6yN1cm/fCukZ1/LM+vJt+X4KGde1Lhi7RRuz19t999b5CXLwt45bd2cf4xr9r2rtTpqBRzZ7p1gXcNWR60pNPf6WUoxvswiouBwW+H/nI6U58N2Us+avoMEYcJC6A/UU9p0DJQQtJMkZzPtFOcuSWMmrhbf9iHcjAHIkzAz5ZXrKfSJSg+z5GCvCnT0T/oQWjVFG4+SdcTECtGIaiNI3SH/AVA8X6K6T2ck85yKIUom6fEaGJZk3TcnvMbaT84XZh4SocHV7zxg24cRJMzHFaa0bXbp0J2lFoBFYB2cdGtdYxH+1o0ApzEedxirhyR2f2K5bZMeprNgiMFfcnj2Jd+qZO3uSZufVHm91tkUe5M1y8A5CP5GdqijdkunJ58t2rbH345jmmQyKJ/6iIVKCQJ9ksnQQAHuT5nbz5v2CSemUToX4Pcg7ZulCTza04wwfHo4AQdQEMrmp+bnEv2NXX7+rwXcHw2pEZdWgW+go8=
2022-04-07T06:36:30.046 INFO:teuthology.kill:  smithi189.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMIjamlOzLpvIDgLEdw+8j6Ce3ATFLm0CiXpH3M/7e3TOtE5wyJgTtH5TEuEUynYVb/42v9I+frifhrj1/jU0zTgSUgIXpB5I0JRwCOZXBi6RWVi7HOPkss6xTAOibYI2tLR6+2xdFOdxl0rl87Bd1vtOgiMqTSuRhVlU75HFljlwc9ThtmC9zx1IK9MAv2iFtv5QhfKpMwSyjAEKDD19m2MdzyVI/hAKJcIfVjGiQg4KhJ1KMWGV5B8vsPbq0gPLZFO9Bt7rEQkS4gNQ7elmhFoGf6bMHzC2/WV4lU2W36FTaw3n4Xagk80a8uZGkvbmmrgkrz7GSrkAwmcI2DHvi5tsPp7DrzXYkCM9CSzgt38r5qk+dq40Bo6leXU3G7flv5O67QGVBYF4OCPInEvjrCOMqNpGIGYKOMcYAq38s8at/KYtTz63InMq+Eu0QpBPq089XiL/MGV+m6PSYnSTVoeQnqZHTEvtYuTcyoUHFkRt/s0JP+auSwQBSXOF3DRU=
2022-04-07T06:36:30.072 INFO:teuthology.kill:2022-04-07 06:36:30,072.072 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-07T06:36:30.075 INFO:teuthology.kill:2022-04-07 06:36:30,075.075 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-04-07T06:36:30.149 INFO:teuthology.kill:2022-04-07 06:36:30,148.148 INFO:teuthology.orchestra.console:Power off smithi166
2022-04-07T06:36:30.164 INFO:teuthology.kill:2022-04-07 06:36:30,164.164 INFO:teuthology.orchestra.console:Power off smithi189
2022-04-07T06:36:38.529 INFO:teuthology.kill:2022-04-07 06:36:38,529.529 INFO:teuthology.orchestra.console:Power off for smithi189 completed
2022-04-07T06:36:38.675 INFO:teuthology.kill:2022-04-07 06:36:38,675.675 INFO:teuthology.lock.ops:unlocked: smithi189.front.sepia.ceph.com
2022-04-07T06:36:42.635 INFO:teuthology.kill:2022-04-07 06:36:42,634.634 INFO:teuthology.orchestra.console:Power off for smithi166 completed
2022-04-07T06:36:42.779 INFO:teuthology.kill:2022-04-07 06:36:42,779.779 INFO:teuthology.lock.ops:unlocked: smithi166.front.sepia.ceph.com
2022-04-07T06:38:43.094 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-04-07T06:38:43.195 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi166 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-04-07_05:07:33-fs-master-testing-default-smithi/6780609
2022-04-07T06:38:43.195 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi189 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-04-07_05:07:33-fs-master-testing-default-smithi/6780609