2022-01-07T15:58:56.865 INFO:root:teuthology version: 1.1.0-95a7d479
2022-01-07T15:58:56.984 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi078.front.sepia.ceph.com' reimaging
2022-01-07T15:58:56.985 INFO:teuthology.lock.ops:Updating [ubuntu@smithi078.front.sepia.ceph.com]: reset os type and version on server
2022-01-07T15:58:56.985 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server
2022-01-07T15:58:57.004 INFO:teuthology.lock.ops:Node 'ubuntu@smithi078.front.sepia.ceph.com' reimaging is complete
2022-01-07T15:58:57.005 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi165.front.sepia.ceph.com' reimaging
2022-01-07T15:58:57.005 INFO:teuthology.lock.ops:Updating [ubuntu@smithi165.front.sepia.ceph.com]: reset os type and version on server
2022-01-07T15:58:57.005 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server
2022-01-07T15:58:57.024 INFO:teuthology.lock.ops:Node 'ubuntu@smithi165.front.sepia.ceph.com' reimaging is complete
2022-01-07T15:58:57.203 INFO:teuthology.provision.fog.smithi078:Scheduling deploy of centos 8.stream
2022-01-07T15:58:57.214 INFO:teuthology.provision.fog.smithi165:Scheduling deploy of centos 8.stream
2022-01-07T15:58:57.555 INFO:teuthology.orchestra.console:Power off smithi078
2022-01-07T15:58:57.570 INFO:teuthology.orchestra.console:Power off smithi165
2022-01-07T15:59:05.935 INFO:teuthology.orchestra.console:Power off for smithi165 completed
2022-01-07T15:59:06.036 INFO:teuthology.orchestra.console:Power on smithi165
2022-01-07T15:59:10.042 INFO:teuthology.orchestra.console:Power off for smithi078 completed
2022-01-07T15:59:10.144 INFO:teuthology.orchestra.console:Power on smithi078
2022-01-07T15:59:14.403 INFO:teuthology.orchestra.console:Power on for smithi165 completed
2022-01-07T15:59:14.504 INFO:teuthology.provision.fog.smithi165:Waiting for deploy to finish
2022-01-07T15:59:18.509 INFO:teuthology.orchestra.console:Power on for smithi078 completed
2022-01-07T15:59:18.611 INFO:teuthology.provision.fog.smithi078:Waiting for deploy to finish
2022-01-07T16:01:45.506 ERROR:teuthology.orchestra.connection:Error authenticating with smithi165.front.sepia.ceph.com: Authentication failed.
2022-01-07T16:03:58.795 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-07T16:03:59.079 INFO:teuthology.provision.fog.smithi165:Node is ready
2022-01-07T16:03:59.097 INFO:teuthology.orchestra.run.smithi165.stdout:smithi165.front.sepia.ceph.com
2022-01-07T16:03:59.155 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.15.165 smithi165.front.sepia.ceph.com smithi165
2022-01-07T16:03:59.487 INFO:teuthology.provision.fog.smithi165:Deploy complete!
2022-01-07T16:04:07.986 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-07T16:04:08.268 INFO:teuthology.provision.fog.smithi078:Node is ready
2022-01-07T16:04:08.286 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078.front.sepia.ceph.com
2022-01-07T16:04:08.345 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78 smithi078.front.sepia.ceph.com smithi078
2022-01-07T16:04:08.690 INFO:teuthology.provision.fog.smithi078:Deploy complete!
2022-01-07T16:04:08.835 INFO:teuthology.lock.ops:Checking smithi078.front.sepia.ceph.com
2022-01-07T16:04:08.836 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-07T16:04:08.855 INFO:teuthology.lock.ops:Checking smithi165.front.sepia.ceph.com
2022-01-07T16:04:08.855 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-07T16:04:08.872 INFO:teuthology.lock.ops:Updating [smithi078.front.sepia.ceph.com]: set os type and version on server
2022-01-07T16:04:08.873 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com'
2022-01-07T16:04:09.057 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64
2022-01-07T16:04:09.112 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="CentOS Stream"
2022-01-07T16:04:09.112 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="8"
2022-01-07T16:04:09.112 INFO:teuthology.orchestra.run.smithi078.stdout:ID="centos"
2022-01-07T16:04:09.113 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="rhel fedora"
2022-01-07T16:04:09.113 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="8"
2022-01-07T16:04:09.113 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el8"
2022-01-07T16:04:09.113 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="CentOS Stream 8"
2022-01-07T16:04:09.114 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31"
2022-01-07T16:04:09.114 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-01-07T16:04:09.114 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://centos.org/"
2022-01-07T16:04:09.114 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-01-07T16:04:09.115 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-01-07T16:04:09.115 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-01-07T16:04:09.117 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server
2022-01-07T16:04:09.134 INFO:teuthology.lock.ops:Updating [smithi165.front.sepia.ceph.com]: set os type and version on server
2022-01-07T16:04:09.135 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com'
2022-01-07T16:04:09.329 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64
2022-01-07T16:04:09.386 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream"
2022-01-07T16:04:09.387 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="8"
2022-01-07T16:04:09.387 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos"
2022-01-07T16:04:09.387 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora"
2022-01-07T16:04:09.387 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="8"
2022-01-07T16:04:09.388 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el8"
2022-01-07T16:04:09.388 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 8"
2022-01-07T16:04:09.388 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31"
2022-01-07T16:04:09.388 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-01-07T16:04:09.389 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/"
2022-01-07T16:04:09.389 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-01-07T16:04:09.389 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-01-07T16:04:09.389 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-01-07T16:04:09.390 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server
2022-01-07T16:04:09.484 INFO:teuthology.dispatcher.supervisor:Running job 6601355
2022-01-07T16:04:09.485 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_95a7d4799b562f3bbb5ec66107094963abd62fa1/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-01-07_15:47:55-rados-wip-yuri8-testing-2022-01-05-1411-distro-default-smithi/6601355 --name yuriw-2022-01-07_15:47:55-rados-wip-yuri8-testing-2022-01-05-1411-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2022-01-07_15:47:55-rados-wip-yuri8-testing-2022-01-05-1411-distro-default-smithi/6601355/orig.config.yaml
2022-01-07T16:04:09.488 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-01-07_15:47:55-rados-wip-yuri8-testing-2022-01-05-1411-distro-default-smithi/6601355
2022-01-07T16:04:09.489 INFO:teuthology.dispatcher.supervisor:Job PID: 30366
2022-01-07T16:04:09.489 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-01-08T04:04:29.564 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-01-08T04:04:29.740 INFO:teuthology.kill:Killing Pids: {30366}
2022-01-08T04:04:30.880 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-01-08T04:04:30.881 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-01-08T04:04:30.881 INFO:teuthology.misc:Compressing logs...
2022-01-08T04:04:30.882 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com'
2022-01-08T04:04:31.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com'
2022-01-08T04:04:31.124 INFO:teuthology.orchestra.run.smithi078.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-01-08T04:04:31.397 INFO:teuthology.orchestra.run.smithi165.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-01-08T04:04:31.443 INFO:teuthology.orchestra.run.smithi078.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-01-08T04:04:31.443 INFO:teuthology.orchestra.run.smithi078.stderr:tar: Error is not recoverable: exiting now
2022-01-08T04:04:31.477 INFO:teuthology.orchestra.run.smithi165.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-01-08T04:04:31.478 INFO:teuthology.orchestra.run.smithi165.stderr:tar: Error is not recoverable: exiting now
2022-01-08T04:04:31.480 INFO:teuthology.misc:Compressing logs...
2022-01-08T04:04:31.767 INFO:teuthology.orchestra.run.smithi165.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-01-08T04:04:31.839 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-01-08T04:04:32.037 INFO:teuthology.misc:Compressing logs...
2022-01-08T04:04:32.653 INFO:teuthology.orchestra.run.smithi165.stderr:gzip: /var/log/ceph/a603e228-6fd4-11ec-8c32-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-01-08T04:04:33.125 INFO:teuthology.orchestra.run.smithi165.stderr:gzip: /var/log/ceph/a603e228-6fd4-11ec-8c32-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-01-08T04:04:33.467 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/a603e228-6fd4-11ec-8c32-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-01-08T04:04:33.672 INFO:teuthology.orchestra.run.smithi165.stderr:gzip: /var/log/ceph/a603e228-6fd4-11ec-8c32-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-01-08T04:04:34.390 INFO:teuthology.orchestra.run.smithi165.stderr:gzip: /var/log/ceph/a603e228-6fd4-11ec-8c32-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-01-08T04:04:34.880 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/a603e228-6fd4-11ec-8c32-001a4aab830c/ceph-mon.smithi078.log: file size changed while zipping
2022-01-08T04:04:36.210 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/a603e228-6fd4-11ec-8c32-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-01-08T04:04:37.193 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/a603e228-6fd4-11ec-8c32-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-01-08T04:04:38.626 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/a603e228-6fd4-11ec-8c32-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-01-08T04:05:26.631 INFO:teuthology.kill:No teuthology processes running
2022-01-08T04:05:27.794 INFO:teuthology.kill:Nuking machines: ['smithi078', 'smithi165']
2022-01-08T04:05:28.733 INFO:teuthology.kill:2022-01-08 04:05:28,733.733 INFO:teuthology.nuke:targets:
2022-01-08T04:05:28.734 INFO:teuthology.kill:  smithi078.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5y5wbrfxV7fFJlxxT9iAxNotEvgmO071Ulgq7c4n00NPoenyJQn6Gnbv2aBAP8kfIGf+1DpEc3ExWX8fH8UBKTpQuX7JiKXnXpqDyngQInShSO0TQsBxbSyAvEeatHvPmfrY/8QpSv5kB2G/acgT2o5oSUtBxf1s8jCGKyJpOd7DG1J6qPY0vRwWSFd2j1NdWSDRFOFWWv+kY/Dfs5zb8AaJAorVXf3eB4U0TRIOYHyZgFFqA7GT+xZl10iSSzi1egfA4qTcZ8Foh+wWQBtGyF2/TjU6FcWcI9OU1rO5zNhY52dFH0taNgvboKTAAOnVW0wCQK43gy5CDUc96II1VjrL+tf+GIwZtZxVwWVEBgckwy/Os6amNPhPr5tQqvMOZfT4lKPthAmuh7C2/xv4ayve6S0ZGCnXx7NFHwhaefSNRf7oFm0Vr4em20bbfrMh/vZrnjNd6cIPtQjNApE9wBWMlX60ah2GMQjsSKfYdI9Fbx6LY6MEZ+XDqtP1uCpE=
2022-01-08T04:05:28.734 INFO:teuthology.kill:  smithi165.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrzCRQ9z/mTdJ1N+CUOcdx5sInLTnpNdSI4pIfrpbbFNhyzS15J63mCGyFwvkS26CbhVUhOQvd3V0X+h+nDnc+nuA8Vmmw/E6f8SxT3CiwbvcjiDJ7josD9SRamEfN5JJqCgTGtcltEm0rcJHyAybPptPy1Kqu5V1IjiyoKoR5HGtp5vhpmnbpKqB/1SDtVqGGfovcceApyPutiZB+nGOfXKoVGCyqKxaAQWO1ngOAkK4KqOE1h719JAH/TaGrsx39zpqIztjVYA7pWm59FLChyqV9otiDZGkKZk7s05I85ho8fI3D5SqmGPQSVKZbEp7I8Nq6UG5PUWGthWAvFmO5ueFKSkYO+dkPSgdEsekBPGvODDoNf1X/aryoCAjzpJvHX96XMHKJcV1q9BBRbbqJzqS0pHIPcXQII0MGN4ZKsd94wczjI2UjsyVRmgYkBTA5/waJ6yCmB2+s2Iw9FtLGzxLVqQn+T5E3unHRCwsmP0NrcErT/fcASIKDxWmFy9c=
2022-01-08T04:05:28.760 INFO:teuthology.kill:2022-01-08 04:05:28,760.760 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-08T04:05:28.764 INFO:teuthology.kill:2022-01-08 04:05:28,763.763 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-08T04:05:28.837 INFO:teuthology.kill:2022-01-08 04:05:28,837.837 INFO:teuthology.orchestra.console:Power off smithi078
2022-01-08T04:05:28.851 INFO:teuthology.kill:2022-01-08 04:05:28,850.850 INFO:teuthology.orchestra.console:Power off smithi165
2022-01-08T04:05:37.209 INFO:teuthology.kill:2022-01-08 04:05:37,208.208 INFO:teuthology.orchestra.console:Power off for smithi078 completed
2022-01-08T04:05:37.214 INFO:teuthology.kill:2022-01-08 04:05:37,214.214 INFO:teuthology.orchestra.console:Power off for smithi165 completed
2022-01-08T04:05:37.358 INFO:teuthology.kill:2022-01-08 04:05:37,357.357 INFO:teuthology.lock.ops:unlocked: smithi165.front.sepia.ceph.com
2022-01-08T04:05:37.362 INFO:teuthology.kill:2022-01-08 04:05:37,362.362 INFO:teuthology.lock.ops:unlocked: smithi078.front.sepia.ceph.com
2022-01-08T04:07:37.585 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-01-08T04:07:37.678 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi078 but it was locked by another job: /home/teuthworker/archive/mrfreezeex-2022-01-07_21:07:03-rbd:mirror-wip-53031-MrFreezeex-octopus-distro-default-smithi/6601989
2022-01-08T04:07:37.678 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi165 but it was locked by another job: /home/teuthworker/archive/mrfreezeex-2022-01-07_21:07:03-rbd:mirror-wip-53031-MrFreezeex-octopus-distro-default-smithi/6601989