2021-12-07T14:19:39.414 INFO:root:teuthology version: 1.1.0-844d1377
2021-12-07T14:19:39.541 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi153.front.sepia.ceph.com' reimaging
2021-12-07T14:19:39.542 INFO:teuthology.lock.ops:Updating [ubuntu@smithi153.front.sepia.ceph.com]: reset os type and version on server
2021-12-07T14:19:39.543 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server
2021-12-07T14:19:39.561 INFO:teuthology.lock.ops:Node 'ubuntu@smithi153.front.sepia.ceph.com' reimaging is complete
2021-12-07T14:19:39.562 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi197.front.sepia.ceph.com' reimaging
2021-12-07T14:19:39.562 INFO:teuthology.lock.ops:Updating [ubuntu@smithi197.front.sepia.ceph.com]: reset os type and version on server
2021-12-07T14:19:39.562 INFO:teuthology.lock.ops:Updating smithi197.front.sepia.ceph.com on lock server
2021-12-07T14:19:39.580 INFO:teuthology.lock.ops:Node 'ubuntu@smithi197.front.sepia.ceph.com' reimaging is complete
2021-12-07T14:19:39.772 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of centos 8.stream
2021-12-07T14:19:39.794 INFO:teuthology.provision.fog.smithi197:Scheduling deploy of centos 8.stream
2021-12-07T14:19:40.166 INFO:teuthology.orchestra.console:Power off smithi153
2021-12-07T14:19:40.233 INFO:teuthology.orchestra.console:Power off smithi197
2021-12-07T14:19:48.539 INFO:teuthology.orchestra.console:Power off for smithi153 completed
2021-12-07T14:19:48.595 INFO:teuthology.orchestra.console:Power off for smithi197 completed
2021-12-07T14:19:48.641 INFO:teuthology.orchestra.console:Power on smithi153
2021-12-07T14:19:48.697 INFO:teuthology.orchestra.console:Power on smithi197
2021-12-07T14:19:57.005 INFO:teuthology.orchestra.console:Power on for smithi153 completed
2021-12-07T14:19:57.077 INFO:teuthology.orchestra.console:Power on for smithi197 completed
2021-12-07T14:19:57.107 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish
2021-12-07T14:19:57.178 INFO:teuthology.provision.fog.smithi197:Waiting for deploy to finish
2021-12-07T14:24:09.171 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-07T14:24:34.412 INFO:teuthology.provision.fog.smithi153:Node is ready
2021-12-07T14:24:34.434 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com
2021-12-07T14:24:34.492 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153
2021-12-07T14:24:34.844 INFO:teuthology.provision.fog.smithi153:Deploy complete!
2021-12-07T14:24:46.696 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-12-07T14:24:46.981 INFO:teuthology.provision.fog.smithi197:Node is ready
2021-12-07T14:24:46.999 INFO:teuthology.orchestra.run.smithi197.stdout:smithi197.front.sepia.ceph.com
2021-12-07T14:24:47.055 INFO:teuthology.orchestra.run.smithi197.stdout:172.21.15.197 smithi197.front.sepia.ceph.com smithi197
2021-12-07T14:24:47.400 INFO:teuthology.provision.fog.smithi197:Deploy complete!
2021-12-07T14:24:47.547 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com
2021-12-07T14:24:47.548 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-07T14:24:47.567 INFO:teuthology.lock.ops:Checking smithi197.front.sepia.ceph.com
2021-12-07T14:24:47.568 INFO:teuthology.lock.ops:New key found. Updating...
2021-12-07T14:24:47.586 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server
2021-12-07T14:24:47.587 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com'
2021-12-07T14:24:47.838 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64
2021-12-07T14:24:47.895 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream"
2021-12-07T14:24:47.896 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8"
2021-12-07T14:24:47.896 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos"
2021-12-07T14:24:47.896 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora"
2021-12-07T14:24:47.896 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8"
2021-12-07T14:24:47.896 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8"
2021-12-07T14:24:47.897 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 8"
2021-12-07T14:24:47.897 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31"
2021-12-07T14:24:47.897 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-07T14:24:47.897 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/"
2021-12-07T14:24:47.898 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-07T14:24:47.898 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-07T14:24:47.898 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-12-07T14:24:47.899 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server
2021-12-07T14:24:47.918 INFO:teuthology.lock.ops:Updating [smithi197.front.sepia.ceph.com]: set os type and version on server
2021-12-07T14:24:47.918 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi197.front.sepia.ceph.com'
2021-12-07T14:24:48.101 INFO:teuthology.orchestra.run.smithi197.stdout:x86_64
2021-12-07T14:24:48.156 INFO:teuthology.orchestra.run.smithi197.stdout:NAME="CentOS Stream"
2021-12-07T14:24:48.156 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION="8"
2021-12-07T14:24:48.156 INFO:teuthology.orchestra.run.smithi197.stdout:ID="centos"
2021-12-07T14:24:48.157 INFO:teuthology.orchestra.run.smithi197.stdout:ID_LIKE="rhel fedora"
2021-12-07T14:24:48.157 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION_ID="8"
2021-12-07T14:24:48.157 INFO:teuthology.orchestra.run.smithi197.stdout:PLATFORM_ID="platform:el8"
2021-12-07T14:24:48.157 INFO:teuthology.orchestra.run.smithi197.stdout:PRETTY_NAME="CentOS Stream 8"
2021-12-07T14:24:48.157 INFO:teuthology.orchestra.run.smithi197.stdout:ANSI_COLOR="0;31"
2021-12-07T14:24:48.158 INFO:teuthology.orchestra.run.smithi197.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-12-07T14:24:48.158 INFO:teuthology.orchestra.run.smithi197.stdout:HOME_URL="https://centos.org/"
2021-12-07T14:24:48.158 INFO:teuthology.orchestra.run.smithi197.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-12-07T14:24:48.158 INFO:teuthology.orchestra.run.smithi197.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2021-12-07T14:24:48.159 INFO:teuthology.orchestra.run.smithi197.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2021-12-07T14:24:48.159 INFO:teuthology.lock.ops:Updating smithi197.front.sepia.ceph.com on lock server
2021-12-07T14:24:48.254 INFO:teuthology.dispatcher.supervisor:Running job 6548067
2021-12-07T14:24:48.254 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6548067 --name benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6548067/orig.config.yaml
2021-12-07T14:24:48.257 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6548067
2021-12-07T14:24:48.258 INFO:teuthology.dispatcher.supervisor:Job PID: 10974
2021-12-07T14:24:48.258 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-12-08T02:25:08.324 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-12-08T02:25:08.518 INFO:teuthology.kill:Killing Pids: {10974}
2021-12-08T02:25:09.759 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2021-12-08T02:25:09.760 INFO:teuthology.task.internal:roles: ubuntu@smithi197.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2021-12-08T02:25:09.760 INFO:teuthology.misc:Compressing logs...
2021-12-08T02:25:09.761 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com'
2021-12-08T02:25:09.956 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi197.front.sepia.ceph.com'
2021-12-08T02:25:10.005 INFO:teuthology.orchestra.run.smithi153.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2021-12-08T02:25:10.193 INFO:teuthology.orchestra.run.smithi197.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2021-12-08T02:25:10.253 INFO:teuthology.orchestra.run.smithi153.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-12-08T02:25:10.254 INFO:teuthology.orchestra.run.smithi153.stderr:tar: Error is not recoverable: exiting now
2021-12-08T02:25:10.299 INFO:teuthology.orchestra.run.smithi197.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2021-12-08T02:25:10.299 INFO:teuthology.orchestra.run.smithi197.stderr:tar: Error is not recoverable: exiting now
2021-12-08T02:25:10.300 INFO:teuthology.misc:Compressing logs...
2021-12-08T02:25:10.601 INFO:teuthology.orchestra.run.smithi197.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-12-08T02:25:10.655 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2021-12-08T02:25:10.850 INFO:teuthology.misc:Compressing logs...
2021-12-08T02:25:30.669 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/f6942366-5769-11ec-8c2e-001a4aab830c/ceph-mon.smithi153.log: file size changed while zipping
2021-12-08T02:25:39.791 INFO:teuthology.orchestra.run.smithi197.stderr:gzip: /var/log/ceph/f6942366-5769-11ec-8c2e-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2021-12-08T02:25:56.311 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/f6942366-5769-11ec-8c2e-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2021-12-08T02:26:03.846 INFO:teuthology.orchestra.run.smithi197.stderr:gzip: /var/log/ceph/f6942366-5769-11ec-8c2e-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2021-12-08T02:26:11.521 INFO:teuthology.orchestra.run.smithi197.stderr:gzip: /var/log/ceph/f6942366-5769-11ec-8c2e-001a4aab830c/ceph-mgr.smithi197.mndshn.log: file size changed while zipping
2021-12-08T02:26:20.396 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/f6942366-5769-11ec-8c2e-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2021-12-08T02:26:36.175 INFO:teuthology.orchestra.run.smithi197.stderr:gzip: /var/log/ceph/f6942366-5769-11ec-8c2e-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2021-12-08T02:26:37.043 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/f6942366-5769-11ec-8c2e-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2021-12-08T02:26:57.373 INFO:teuthology.orchestra.run.smithi197.stderr:gzip: /var/log/ceph/f6942366-5769-11ec-8c2e-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2021-12-08T02:27:02.564 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /var/log/ceph/f6942366-5769-11ec-8c2e-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2021-12-08T02:27:07.242 INFO:teuthology.orchestra.run.smithi197.stderr:gzip: /var/log/ceph/f6942366-5769-11ec-8c2e-001a4aab830c/ceph-mon.smithi197.log: file size changed while zipping
2021-12-08T02:27:55.003 INFO:teuthology.kill:No teuthology processes running
2021-12-08T02:27:56.232 INFO:teuthology.kill:Nuking machines: ['smithi153', 'smithi197']
2021-12-08T02:27:57.194 INFO:teuthology.kill:2021-12-08 02:27:57,194.194 INFO:teuthology.nuke:targets:
2021-12-08T02:27:57.195 INFO:teuthology.kill:  smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDF+iAfrab9XXnO6lZYo5CgjSDJgzJ7bYuYf7GTcDrLIxfSPbrrmwZgMRkEFP4T4+/KZ+lYZh7TasnKCT82iX9aFZJtw6/V9/trsFslEPbdcq8qgBkXSEeBlEaTnUt7Sw9fthkCpGFVTq6qlYlIqzSd+ZChVaoSS8yLP6201dvJW8mjBMALUz3r7ytZdOUeQyc8voxEqFD/0sX2riN/eRqpKVlzE/YhflaZHuCCzz9/qdxbdbvkFxZ6CLVOGkujiRZeCtl/5uwF27ew2hG+czdT1Rjxlrkz5LxAtTG8EKIPAyUqZluzHCEOzFaqk+X5Au6i1W0N6EQTOERMt2jHBsWat2sJv3eiCybePMVaTixw6uo7wRfoQdJsAEWY3AozoUkEu3Uw4V2mZQ7JFvFPnOaIAvEvpAB+crKKfF4oryOI2oUl2JHJmWi5AK2fzLvu9IBt/3vKWCGY6MF3vVj3TymBPeYAVRZkq6L9GrcEUAhEPWur+niDwXBJdiZ9N9/c1qU=
2021-12-08T02:27:57.195 INFO:teuthology.kill:  smithi197.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWNlFD4nW3uwO+g0ZfOJlLy3Wz3XZv+IGUx/3Wqra3mBMqBVlPIGNppsmH465EJ/EuUSwvqBE42jicdsQLYwFtDvJgUQbh2GpK5jCm8kUFXM7FSOK90h+fmdT78nGQQj02p84LYDInmtTYiFCoy7d8Hu0oUlXuOAQcJ1yvK1Pvlhmcs61P7obAZh2e2W0/wqdF6zMZW6SW7T6UbKZVaEOCwv9CCXIEm7lv0MMiOideeo3pw9I/ladGAMdb1FgQsZoybazyhqqSdIRYtMjBlwxh7egVdMore6+zObyRsygJ8IdMyDogIoKNDtu3o2/+uQp2lCPYeb3Ah7M9S21UZLQrEsEwnYKa5Vlc45WQB3uzeeXwbauvqRBDkrdUBSWQtWU3CzdYgP32JyQtBEwqV/R2OPvEqXoaDd+HC9tZ59RJI94PieMCseN4l7Xd0bAuG1PQQ8SvTpkADMcV/iO7cgMrzyGQ+aelHSkZNmtx4xn2wrKcTEutIyfs6MfggUE7JWc=
2021-12-08T02:27:57.238 INFO:teuthology.kill:2021-12-08 02:27:57,237.237 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-08T02:27:57.241 INFO:teuthology.kill:2021-12-08 02:27:57,241.241 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-12-08T02:27:57.332 INFO:teuthology.kill:2021-12-08 02:27:57,331.331 INFO:teuthology.orchestra.console:Power off smithi197
2021-12-08T02:27:57.348 INFO:teuthology.kill:2021-12-08 02:27:57,347.347 INFO:teuthology.orchestra.console:Power off smithi153
2021-12-08T02:28:09.926 INFO:teuthology.kill:2021-12-08 02:28:09,926.926 INFO:teuthology.orchestra.console:Power off for smithi197 completed
2021-12-08T02:28:09.933 INFO:teuthology.kill:2021-12-08 02:28:09,933.933 INFO:teuthology.orchestra.console:Power off for smithi153 completed
2021-12-08T02:28:10.072 INFO:teuthology.kill:2021-12-08 02:28:10,071.071 INFO:teuthology.lock.ops:unlocked: smithi197.front.sepia.ceph.com
2021-12-08T02:28:10.076 INFO:teuthology.kill:2021-12-08 02:28:10,076.076 INFO:teuthology.lock.ops:unlocked: smithi153.front.sepia.ceph.com
2021-12-08T02:30:10.383 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-12-08T02:30:10.483 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi153 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-07_16:02:55-rados-wip-yuri11-testing-2021-12-06-1619-distro-default-smithi/6551080
2021-12-08T02:30:10.483 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi197 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-12-07_16:02:55-rados-wip-yuri11-testing-2021-12-06-1619-distro-default-smithi/6551080