2022-03-06T11:17:42.215 INFO:root:teuthology version: 1.1.0-eeaeee38
2022-03-06T11:17:42.335 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi179.front.sepia.ceph.com' reimaging
2022-03-06T11:17:42.336 INFO:teuthology.lock.ops:Updating [ubuntu@smithi179.front.sepia.ceph.com]: reset os type and version on server
2022-03-06T11:17:42.337 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server
2022-03-06T11:17:42.355 INFO:teuthology.lock.ops:Node 'ubuntu@smithi179.front.sepia.ceph.com' reimaging is complete
2022-03-06T11:17:42.356 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi181.front.sepia.ceph.com' reimaging
2022-03-06T11:17:42.356 INFO:teuthology.lock.ops:Updating [ubuntu@smithi181.front.sepia.ceph.com]: reset os type and version on server
2022-03-06T11:17:42.356 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server
2022-03-06T11:17:42.374 INFO:teuthology.lock.ops:Node 'ubuntu@smithi181.front.sepia.ceph.com' reimaging is complete
2022-03-06T11:17:42.566 INFO:teuthology.provision.fog.smithi179:Scheduling deploy of centos 8.stream
2022-03-06T11:17:42.575 INFO:teuthology.provision.fog.smithi181:Scheduling deploy of centos 8.stream
2022-03-06T11:17:42.919 INFO:teuthology.orchestra.console:Power off smithi181
2022-03-06T11:17:42.938 INFO:teuthology.orchestra.console:Power off smithi179
2022-03-06T11:17:51.301 INFO:teuthology.orchestra.console:Power off for smithi181 completed
2022-03-06T11:17:51.304 INFO:teuthology.orchestra.console:Power off for smithi179 completed
2022-03-06T11:17:51.403 INFO:teuthology.orchestra.console:Power on smithi181
2022-03-06T11:17:51.411 INFO:teuthology.orchestra.console:Power on smithi179
2022-03-06T11:17:59.774 INFO:teuthology.orchestra.console:Power on for smithi181 completed
2022-03-06T11:17:59.778 INFO:teuthology.orchestra.console:Power on for smithi179 completed
2022-03-06T11:17:59.876 INFO:teuthology.provision.fog.smithi181:Waiting for deploy to finish
2022-03-06T11:17:59.879 INFO:teuthology.provision.fog.smithi179:Waiting for deploy to finish
2022-03-06T11:21:59.183 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-06T11:22:24.524 INFO:teuthology.provision.fog.smithi181:Node is ready
2022-03-06T11:22:24.589 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181.front.sepia.ceph.com
2022-03-06T11:22:24.648 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181 smithi181.front.sepia.ceph.com smithi181
2022-03-06T11:22:25.029 INFO:teuthology.provision.fog.smithi181:Deploy complete!
2022-03-06T11:22:31.768 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-06T11:22:31.996 INFO:teuthology.provision.fog.smithi179:Node is ready
2022-03-06T11:22:32.055 INFO:teuthology.orchestra.run.smithi179.stdout:smithi179.front.sepia.ceph.com
2022-03-06T11:22:32.114 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179 smithi179.front.sepia.ceph.com smithi179
2022-03-06T11:22:32.461 INFO:teuthology.provision.fog.smithi179:Deploy complete!
2022-03-06T11:22:32.608 INFO:teuthology.lock.ops:Checking smithi179.front.sepia.ceph.com
2022-03-06T11:22:32.608 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-06T11:22:32.627 INFO:teuthology.lock.ops:Checking smithi181.front.sepia.ceph.com
2022-03-06T11:22:32.628 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-06T11:22:32.649 INFO:teuthology.lock.ops:Updating [smithi179.front.sepia.ceph.com]: set os type and version on server
2022-03-06T11:22:32.650 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com'
2022-03-06T11:22:32.839 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64
2022-03-06T11:22:32.895 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream"
2022-03-06T11:22:32.896 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="8"
2022-03-06T11:22:32.896 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos"
2022-03-06T11:22:32.896 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora"
2022-03-06T11:22:32.896 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="8"
2022-03-06T11:22:32.897 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el8"
2022-03-06T11:22:32.897 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 8"
2022-03-06T11:22:32.897 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31"
2022-03-06T11:22:32.897 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-03-06T11:22:32.897 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/"
2022-03-06T11:22:32.898 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-06T11:22:32.898 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-06T11:22:32.898 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-03-06T11:22:32.899 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server
2022-03-06T11:22:32.916 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: set os type and version on server
2022-03-06T11:22:32.917 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2022-03-06T11:22:33.113 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64
2022-03-06T11:22:33.172 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream"
2022-03-06T11:22:33.172 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8"
2022-03-06T11:22:33.173 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos"
2022-03-06T11:22:33.173 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora"
2022-03-06T11:22:33.173 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8"
2022-03-06T11:22:33.174 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8"
2022-03-06T11:22:33.174 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 8"
2022-03-06T11:22:33.174 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31"
2022-03-06T11:22:33.174 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-03-06T11:22:33.175 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/"
2022-03-06T11:22:33.175 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-06T11:22:33.175 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-06T11:22:33.175 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-03-06T11:22:33.176 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server
2022-03-06T11:22:33.270 INFO:teuthology.dispatcher.supervisor:Running job 6723445
2022-03-06T11:22:33.271 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_eeaeee38264fad8a01fd1ce912d8908676ed64bc/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-03-06_03:31:03-rados-pacific-distro-default-smithi/6723445 --name teuthology-2022-03-06_03:31:03-rados-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/teuthology-2022-03-06_03:31:03-rados-pacific-distro-default-smithi/6723445/orig.config.yaml
2022-03-06T11:22:33.274 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-03-06_03:31:03-rados-pacific-distro-default-smithi/6723445
2022-03-06T11:22:33.275 INFO:teuthology.dispatcher.supervisor:Job PID: 18120
2022-03-06T11:22:33.275 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-03-06T17:53:02.703 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-03-06T17:53:02.843 INFO:teuthology.kill:Killing Pids: {18120}
2022-03-06T17:53:03.989 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-03-06T17:53:03.991 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-03-06T17:53:03.991 INFO:teuthology.misc:Compressing logs...
2022-03-06T17:53:03.992 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com'
2022-03-06T17:53:04.185 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com'
2022-03-06T17:53:04.235 INFO:teuthology.orchestra.run.smithi179.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-03-06T17:53:04.427 INFO:teuthology.orchestra.run.smithi181.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-03-06T17:53:04.470 INFO:teuthology.orchestra.run.smithi179.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-03-06T17:53:04.470 INFO:teuthology.orchestra.run.smithi179.stderr:tar: Error is not recoverable: exiting now
2022-03-06T17:53:04.511 INFO:teuthology.orchestra.run.smithi181.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-03-06T17:53:04.511 INFO:teuthology.orchestra.run.smithi181.stderr:tar: Error is not recoverable: exiting now
2022-03-06T17:53:04.512 INFO:teuthology.misc:Compressing logs...
2022-03-06T17:53:04.917 INFO:teuthology.misc:Compressing logs...
2022-03-06T17:53:08.484 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/daaca20a-9d40-11ec-8c35-001a4aab830c/ceph-mon.smithi181.log: file size changed while zipping
2022-03-06T17:53:20.530 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/daaca20a-9d40-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-03-06T17:53:25.091 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/daaca20a-9d40-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-03-06T17:53:32.392 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/daaca20a-9d40-11ec-8c35-001a4aab830c/ceph-mon.smithi179.log: file size changed while zipping
2022-03-06T17:53:43.193 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/daaca20a-9d40-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-03-06T17:53:44.969 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/daaca20a-9d40-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-03-06T17:53:45.780 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/daaca20a-9d40-11ec-8c35-001a4aab830c/ceph-mgr.smithi179.tqqhyv.log: file size changed while zipping
2022-03-06T17:53:59.003 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/daaca20a-9d40-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-03-06T17:54:03.006 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/daaca20a-9d40-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-03-06T17:54:14.841 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/daaca20a-9d40-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-03-06T17:54:20.482 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/daaca20a-9d40-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-03-06T17:54:50.892 INFO:teuthology.kill:No teuthology processes running
2022-03-06T17:54:52.147 INFO:teuthology.kill:Nuking machines: ['smithi179', 'smithi181']
2022-03-06T17:54:53.108 INFO:teuthology.kill:2022-03-06 17:54:53,108.108 INFO:teuthology.nuke:targets:
2022-03-06T17:54:53.109 INFO:teuthology.kill:  smithi179.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/rd53z1repEvLPW5hGEARW8i8xhe6YI1v3uybejItRKeQefLoTb46IMXNrzf0SSvkLtVg6X1DxuRGgWsCjEmDv6HDdhCH9kN2LY81AzZpMuJNJ+MixsvL8fb7m+L4bhPuQA0xVp29LJ0jo6p3B/9crkLo33yQQTMNiO/S5gBXkgte8xMm5zN8FCKGUicmP7xIrQDuEnHg3SJyvZX7GcBUH2XnPpCKrdjSgMqvBR8bGRYNZKjn8e4Tmoin8CgGiivrd3FnYSvbbDJGkCtdLHF33FiDUSqd4lIfnpxAzeaTYLZJqfB7rttKAP6ypK7xUohV8YKmkPWrK8y0c3oGv7HUzNG0qAISIYtZaf4wBD/3piB8fIMaTKM8IjXgh9YCKzMYflv+ktrk1j0Ne/NDwDgfqRI5pzmNDitsuW7Z4IdBhS+ZxPVSW/b5lgu0hUlOsxPai1NMSDl3C6MSZrTiu67JHnyxI5S2gpmAzIh/C7vQp3a6sIj0faCYevB9VbypzBU=
2022-03-06T17:54:53.109 INFO:teuthology.kill:  smithi181.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC10RxAYZRYP8H6+n+UtKfQnxdUod31LiWrKKEASGJD4W9UoWbVktkgkWAzzPOAInlqjLiocUjgr2nPZRHGZOl2xqAmRFggA87cWxOxwqSUj+jNnvvqgqSDQALZSK4sP71MLDZdskQ4XTLvGqU8B4zZZrkQfmDE1y8ZGYo8N3rH7fruwSXtEks9ERmNa+FJZultZnXQO91ASj6UPVtRuCzwNPcnXiyxc8hcM4FXhknPWltKdLjnF4gVYcp926NvNJnYOAMxiYHd9wfgv5fMY6BF3RjfFr/EOQWhK8qCpnW+XlQ2KDl0OUzmav58aauXXFAQWh801AWBK0Ia/SUtbjUsORaFAJfEc+LZOlLXiQ3rLd7LctboZbMkl7rguiIfan48oFo/c8U9Ii9jEayC1fmUWTJd4cWHzFF1PsDVmuYEmHLPrd7iTAJPnjgGSs0oJsS9WXqzQU+mBDJC5+TtwWLL93zR/JMh1PBgusWwzp8Rocb9Uhk6YLajTZdSAGYnSbc=
2022-03-06T17:54:53.135 INFO:teuthology.kill:2022-03-06 17:54:53,134.134 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-06T17:54:53.137 INFO:teuthology.kill:2022-03-06 17:54:53,137.137 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-06T17:54:53.221 INFO:teuthology.kill:2022-03-06 17:54:53,220.220 INFO:teuthology.orchestra.console:Power off smithi181
2022-03-06T17:54:53.234 INFO:teuthology.kill:2022-03-06 17:54:53,234.234 INFO:teuthology.orchestra.console:Power off smithi179
2022-03-06T17:55:01.586 INFO:teuthology.kill:2022-03-06 17:55:01,585.585 INFO:teuthology.orchestra.console:Power off for smithi181 completed
2022-03-06T17:55:01.732 INFO:teuthology.kill:2022-03-06 17:55:01,731.731 INFO:teuthology.lock.ops:unlocked: smithi181.front.sepia.ceph.com
2022-03-06T17:55:05.705 INFO:teuthology.kill:2022-03-06 17:55:05,705.705 INFO:teuthology.orchestra.console:Power off for smithi179 completed
2022-03-06T17:55:05.857 INFO:teuthology.kill:2022-03-06 17:55:05,857.857 INFO:teuthology.lock.ops:unlocked: smithi179.front.sepia.ceph.com
2022-03-06T17:57:06.177 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-03-06T17:57:06.287 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi179 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-02-28_05:05:03-rgw-octopus-distro-default-smithi/6710477
2022-03-06T17:57:06.287 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi181 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-02-28_05:05:03-rgw-octopus-distro-default-smithi/6710477