2022-02-08T19:14:59.746 INFO:root:teuthology version: 1.1.0-7cebb3f2
2022-02-08T19:14:59.882 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi105.front.sepia.ceph.com' reimaging
2022-02-08T19:14:59.884 INFO:teuthology.lock.ops:Updating [ubuntu@smithi105.front.sepia.ceph.com]: reset os type and version on server
2022-02-08T19:14:59.884 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server
2022-02-08T19:14:59.904 INFO:teuthology.lock.ops:Node 'ubuntu@smithi105.front.sepia.ceph.com' reimaging is complete
2022-02-08T19:14:59.905 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi179.front.sepia.ceph.com' reimaging
2022-02-08T19:14:59.905 INFO:teuthology.lock.ops:Updating [ubuntu@smithi179.front.sepia.ceph.com]: reset os type and version on server
2022-02-08T19:14:59.906 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server
2022-02-08T19:14:59.925 INFO:teuthology.lock.ops:Node 'ubuntu@smithi179.front.sepia.ceph.com' reimaging is complete
2022-02-08T19:15:00.118 INFO:teuthology.provision.fog.smithi105:Scheduling deploy of centos 8.stream
2022-02-08T19:15:00.126 INFO:teuthology.provision.fog.smithi179:Scheduling deploy of centos 8.stream
2022-02-08T19:15:00.479 INFO:teuthology.orchestra.console:Power off smithi105
2022-02-08T19:15:00.494 INFO:teuthology.orchestra.console:Power off smithi179
2022-02-08T19:15:08.859 INFO:teuthology.orchestra.console:Power off for smithi179 completed
2022-02-08T19:15:08.960 INFO:teuthology.orchestra.console:Power on smithi179
2022-02-08T19:15:12.965 INFO:teuthology.orchestra.console:Power off for smithi105 completed
2022-02-08T19:15:13.067 INFO:teuthology.orchestra.console:Power on smithi105
2022-02-08T19:15:17.326 INFO:teuthology.orchestra.console:Power on for smithi105 completed
2022-02-08T19:15:17.427 INFO:teuthology.provision.fog.smithi105:Waiting for deploy to finish
2022-02-08T19:15:17.455 INFO:teuthology.orchestra.console:Power on for smithi179 completed
2022-02-08T19:15:17.557 INFO:teuthology.provision.fog.smithi179:Waiting for deploy to finish
2022-02-08T19:17:48.767 ERROR:teuthology.orchestra.connection:Error authenticating with smithi179.front.sepia.ceph.com: Authentication failed.
2022-02-08T19:19:29.431 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-08T19:19:32.028 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-08T19:19:55.056 INFO:teuthology.provision.fog.smithi179:Node is ready
2022-02-08T19:19:55.079 INFO:teuthology.orchestra.run.smithi179.stdout:smithi179.front.sepia.ceph.com
2022-02-08T19:19:55.138 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179 smithi179.front.sepia.ceph.com smithi179
2022-02-08T19:19:55.507 INFO:teuthology.provision.fog.smithi179:Deploy complete!
2022-02-08T19:19:57.606 INFO:teuthology.provision.fog.smithi105:Node is ready
2022-02-08T19:19:57.628 INFO:teuthology.orchestra.run.smithi105.stdout:smithi105.front.sepia.ceph.com
2022-02-08T19:19:57.688 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105 smithi105.front.sepia.ceph.com smithi105
2022-02-08T19:19:58.060 INFO:teuthology.provision.fog.smithi105:Deploy complete!
2022-02-08T19:19:58.211 INFO:teuthology.lock.ops:Checking smithi105.front.sepia.ceph.com
2022-02-08T19:19:58.212 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-08T19:19:58.231 INFO:teuthology.lock.ops:Checking smithi179.front.sepia.ceph.com
2022-02-08T19:19:58.231 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-08T19:19:58.248 INFO:teuthology.lock.ops:Updating [smithi105.front.sepia.ceph.com]: set os type and version on server
2022-02-08T19:19:58.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com'
2022-02-08T19:19:58.440 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64
2022-02-08T19:19:58.497 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream"
2022-02-08T19:19:58.498 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="8"
2022-02-08T19:19:58.498 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos"
2022-02-08T19:19:58.499 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora"
2022-02-08T19:19:58.499 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="8"
2022-02-08T19:19:58.500 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el8"
2022-02-08T19:19:58.500 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-08T19:19:58.501 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31"
2022-02-08T19:19:58.501 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-08T19:19:58.502 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/"
2022-02-08T19:19:58.502 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-08T19:19:58.503 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-08T19:19:58.503 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-08T19:19:58.506 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server
2022-02-08T19:19:58.527 INFO:teuthology.lock.ops:Updating [smithi179.front.sepia.ceph.com]: set os type and version on server
2022-02-08T19:19:58.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com'
2022-02-08T19:19:58.731 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64
2022-02-08T19:19:58.792 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream"
2022-02-08T19:19:58.793 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="8"
2022-02-08T19:19:58.793 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos"
2022-02-08T19:19:58.793 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora"
2022-02-08T19:19:58.793 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="8"
2022-02-08T19:19:58.793 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el8"
2022-02-08T19:19:58.794 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-08T19:19:58.794 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31"
2022-02-08T19:19:58.794 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-08T19:19:58.795 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/"
2022-02-08T19:19:58.795 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-08T19:19:58.795 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-08T19:19:58.795 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-08T19:19:58.796 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server
2022-02-08T19:19:58.908 INFO:teuthology.dispatcher.supervisor:Running job 6670286
2022-02-08T19:19:58.908 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-02-08_17:00:23-rados-wip-yuri5-testing-2022-02-08-0733-pacific-distro-default-smithi/6670286 --name yuriw-2022-02-08_17:00:23-rados-wip-yuri5-testing-2022-02-08-0733-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2022-02-08_17:00:23-rados-wip-yuri5-testing-2022-02-08-0733-pacific-distro-default-smithi/6670286/orig.config.yaml
2022-02-08T19:19:58.912 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-02-08_17:00:23-rados-wip-yuri5-testing-2022-02-08-0733-pacific-distro-default-smithi/6670286
2022-02-08T19:19:58.913 INFO:teuthology.dispatcher.supervisor:Job PID: 20866
2022-02-08T19:19:58.914 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-09T01:50:10.237 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-09T01:50:10.403 INFO:teuthology.kill:Killing Pids: {20866}
2022-02-09T01:50:11.587 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-02-09T01:50:11.588 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-02-09T01:50:11.589 INFO:teuthology.misc:Compressing logs...
2022-02-09T01:50:11.589 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com'
2022-02-09T01:50:11.781 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com'
2022-02-09T01:50:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-09T01:50:12.015 INFO:teuthology.orchestra.run.smithi179.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-09T01:50:12.068 INFO:teuthology.orchestra.run.smithi105.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-09T01:50:12.069 INFO:teuthology.orchestra.run.smithi105.stderr:tar: Error is not recoverable: exiting now
2022-02-09T01:50:12.202 INFO:teuthology.orchestra.run.smithi179.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-09T01:50:12.203 INFO:teuthology.orchestra.run.smithi179.stderr:tar: Error is not recoverable: exiting now
2022-02-09T01:50:12.204 INFO:teuthology.misc:Compressing logs...
2022-02-09T01:50:12.452 INFO:teuthology.orchestra.run.smithi105.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-09T01:50:12.611 INFO:teuthology.misc:Compressing logs...
2022-02-09T01:50:16.127 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/8a8a15fe-8914-11ec-8c35-001a4aab830c/ceph-mon.smithi179.log: file size changed while zipping
2022-02-09T01:50:35.228 INFO:teuthology.orchestra.run.smithi105.stderr:gzip: /var/log/ceph/8a8a15fe-8914-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-02-09T01:50:42.145 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/8a8a15fe-8914-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-02-09T01:50:51.054 INFO:teuthology.orchestra.run.smithi105.stderr:gzip: /var/log/ceph/8a8a15fe-8914-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-02-09T01:51:03.148 INFO:teuthology.orchestra.run.smithi105.stderr:gzip: /var/log/ceph/8a8a15fe-8914-11ec-8c35-001a4aab830c/ceph-mon.smithi105.log: file size changed while zipping
2022-02-09T01:51:06.643 INFO:teuthology.orchestra.run.smithi105.stderr:gzip: /var/log/ceph/8a8a15fe-8914-11ec-8c35-001a4aab830c/ceph-mgr.smithi105.ogckyx.log: file size changed while zipping
2022-02-09T01:51:13.875 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/8a8a15fe-8914-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-02-09T01:51:33.022 INFO:teuthology.orchestra.run.smithi105.stderr:gzip: /var/log/ceph/8a8a15fe-8914-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-02-09T01:51:35.661 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/8a8a15fe-8914-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-02-09T01:51:57.468 INFO:teuthology.orchestra.run.smithi179.stderr:gzip: /var/log/ceph/8a8a15fe-8914-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-02-09T01:51:58.936 INFO:teuthology.orchestra.run.smithi105.stderr:gzip: /var/log/ceph/8a8a15fe-8914-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-02-09T01:52:42.303 INFO:teuthology.kill:No teuthology processes running
2022-02-09T01:52:43.480 INFO:teuthology.kill:Nuking machines: ['smithi105', 'smithi179']
2022-02-09T01:52:44.480 INFO:teuthology.kill:2022-02-09 01:52:44,479.479 INFO:teuthology.nuke:targets:
2022-02-09T01:52:44.480 INFO:teuthology.kill:  smithi105.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6htn3obEf2/dbrKFP3QIs5MLOArvQuBsoJeJWK6NKqGTHEvMBb8QKI01v/l7Ch5yHAESCt2XOcGFsh3HHJ7ZUqALH8fqIWmp0q+wFeEynBxbtFtO7RXuS8bAPC/RdFExssQapCCGt+adE/pyEwRFij7K4UeIwmiSwDKXv1Et5JthUlHJWfacDgBZN33a/JriZj4ZFeBDhwLAU2m8AJbUyim5+xh1syWqWA7W2vCe0jSgOlHUjYc0DGURuxEFQQghxIBozuCcwesHRGkctq0E6XtV6YckAsU8WFUoo7YD+M6pvzCF3sFHBTXpghi1WQZejwDGJgBY2BjS/CNf8xT+aEYUQoJnbtsPvjXiXz8GKDDzujfAr1uR4qHsS7R5VjuQ1aGFjI+p9cbn9clxBmXWjBU1TFVHkeDLcn18TwSi6E9tohmwTqndejcOO1B4BUepVEb8h5+Edq+Dt3LhGwaj1NDNuKv08IU2CTeIQ/mVosUPoOjkNlL70dKHYVW97jHk=
2022-02-09T01:52:44.481 INFO:teuthology.kill:  smithi179.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsrDQFW95omCc1Jc7UKQYNQxUTwFWOuNAv39RFSqg/XoA3SR9mIfju6M1uZJDWHmh7QFIY74aygImKY2TLbPTgl4OoMEuaBciK/d7AeYJge/YxOCeB6irMro9inkr8dlT6gLbb9sy+cVQ1lmIbvqKCXZLeePs2wwgp3IC0TDHz8u4/oXj2KS+WjzzSgSsHcsQzf0bNx59cATxUwkecqKTpPHxhXJfFKKis0DfJiFuB3PAvBYYiDWrCAgOqv5o+aDB+blUW3UZbi1AVUCmnGudbcUnwCoLg2bLza1OukMJ1RpCED/ItQhho2MxMgnSLPGGM9bwY0OlC7aFCqksFn5HQHQaXl4j7eXZyVgJUikmAFJQA5VI+0/K1cIeBbJEYLpNX69ZPZbZbOGsTQQXdajAl7CXzEGT4SSG018KeUqKVe3rwEQnOvBgVesqxUVQWZhqlUGw5KXiBGZHG+zWAjfDr1bnJjdfSoW1WimMncZH2X1nwJ66SwV9Sz5uHb3s4SCs=
2022-02-09T01:52:44.507 INFO:teuthology.kill:2022-02-09 01:52:44,507.507 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-09T01:52:44.512 INFO:teuthology.kill:2022-02-09 01:52:44,512.512 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-09T01:52:44.591 INFO:teuthology.kill:2022-02-09 01:52:44,591.591 INFO:teuthology.orchestra.console:Power off smithi105
2022-02-09T01:52:44.609 INFO:teuthology.kill:2022-02-09 01:52:44,609.609 INFO:teuthology.orchestra.console:Power off smithi179
2022-02-09T01:52:52.969 INFO:teuthology.kill:2022-02-09 01:52:52,969.969 INFO:teuthology.orchestra.console:Power off for smithi105 completed
2022-02-09T01:52:53.114 INFO:teuthology.kill:2022-02-09 01:52:53,113.113 INFO:teuthology.lock.ops:unlocked: smithi105.front.sepia.ceph.com
2022-02-09T01:52:57.087 INFO:teuthology.kill:2022-02-09 01:52:57,087.087 INFO:teuthology.orchestra.console:Power off for smithi179 completed
2022-02-09T01:52:57.232 INFO:teuthology.kill:2022-02-09 01:52:57,232.232 INFO:teuthology.lock.ops:unlocked: smithi179.front.sepia.ceph.com
2022-02-09T01:54:57.460 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-09T01:54:57.565 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi105 but it was locked by another job: /home/teuthworker/archive/benhanokh-2022-02-08_14:21:50-rados-WIP_GBH_safe_shutdown_v2C-distro-basic-smithi/6669645
2022-02-09T01:54:57.566 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi179 but it was locked by another job: /home/teuthworker/archive/benhanokh-2022-02-08_14:21:50-rados-WIP_GBH_safe_shutdown_v2C-distro-basic-smithi/6669645