2022-02-13T06:53:41.811 INFO:root:teuthology version: 1.1.0-7cebb3f2
2022-02-13T06:53:41.936 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi079.front.sepia.ceph.com' reimaging
2022-02-13T06:53:41.937 INFO:teuthology.lock.ops:Updating [ubuntu@smithi079.front.sepia.ceph.com]: reset os type and version on server
2022-02-13T06:53:41.938 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server
2022-02-13T06:53:41.956 INFO:teuthology.lock.ops:Node 'ubuntu@smithi079.front.sepia.ceph.com' reimaging is complete
2022-02-13T06:53:41.956 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi100.front.sepia.ceph.com' reimaging
2022-02-13T06:53:41.957 INFO:teuthology.lock.ops:Updating [ubuntu@smithi100.front.sepia.ceph.com]: reset os type and version on server
2022-02-13T06:53:41.957 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server
2022-02-13T06:53:41.975 INFO:teuthology.lock.ops:Node 'ubuntu@smithi100.front.sepia.ceph.com' reimaging is complete
2022-02-13T06:53:42.183 INFO:teuthology.provision.fog.smithi079:Scheduling deploy of centos 8.stream
2022-02-13T06:53:42.187 INFO:teuthology.provision.fog.smithi100:Scheduling deploy of centos 8.stream
2022-02-13T06:53:42.607 INFO:teuthology.orchestra.console:Power off smithi100
2022-02-13T06:53:42.711 INFO:teuthology.orchestra.console:Power off smithi079
2022-02-13T06:53:42.851 INFO:teuthology.orchestra.console:Power off for smithi079 completed
2022-02-13T06:53:42.953 INFO:teuthology.orchestra.console:Power on smithi079
2022-02-13T06:53:50.975 INFO:teuthology.orchestra.console:Power off for smithi100 completed
2022-02-13T06:53:51.076 INFO:teuthology.orchestra.console:Power on smithi100
2022-02-13T06:53:51.316 INFO:teuthology.orchestra.console:Power on for smithi079 completed
2022-02-13T06:53:51.417 INFO:teuthology.provision.fog.smithi079:Waiting for deploy to finish
2022-02-13T06:53:59.441 INFO:teuthology.orchestra.console:Power on for smithi100 completed
2022-02-13T06:53:59.543 INFO:teuthology.provision.fog.smithi100:Waiting for deploy to finish
2022-02-13T06:56:31.083 ERROR:teuthology.orchestra.connection:Error authenticating with smithi100.front.sepia.ceph.com: Authentication failed.
2022-02-13T06:57:57.990 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-13T06:58:11.790 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-13T06:58:23.701 INFO:teuthology.provision.fog.smithi079:Node is ready
2022-02-13T06:58:23.723 INFO:teuthology.orchestra.run.smithi079.stdout:smithi079.front.sepia.ceph.com
2022-02-13T06:58:23.782 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79 smithi079.front.sepia.ceph.com smithi079
2022-02-13T06:58:24.155 INFO:teuthology.provision.fog.smithi079:Deploy complete!
2022-02-13T06:58:36.942 INFO:teuthology.provision.fog.smithi100:Node is ready
2022-02-13T06:58:36.965 INFO:teuthology.orchestra.run.smithi100.stdout:smithi100.front.sepia.ceph.com
2022-02-13T06:58:37.023 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100 smithi100.front.sepia.ceph.com smithi100
2022-02-13T06:58:37.394 INFO:teuthology.provision.fog.smithi100:Deploy complete!
2022-02-13T06:58:37.581 INFO:teuthology.lock.ops:Checking smithi079.front.sepia.ceph.com
2022-02-13T06:58:37.583 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-13T06:58:37.603 INFO:teuthology.lock.ops:Checking smithi100.front.sepia.ceph.com
2022-02-13T06:58:37.603 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-13T06:58:37.621 INFO:teuthology.lock.ops:Updating [smithi079.front.sepia.ceph.com]: set os type and version on server
2022-02-13T06:58:37.622 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com'
2022-02-13T06:58:37.872 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64
2022-02-13T06:58:37.930 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream"
2022-02-13T06:58:37.931 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="8"
2022-02-13T06:58:37.931 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos"
2022-02-13T06:58:37.931 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora"
2022-02-13T06:58:37.931 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="8"
2022-02-13T06:58:37.932 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el8"
2022-02-13T06:58:37.932 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-13T06:58:37.932 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31"
2022-02-13T06:58:37.932 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-13T06:58:37.932 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/"
2022-02-13T06:58:37.933 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-13T06:58:37.933 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-13T06:58:37.933 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-13T06:58:37.934 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server
2022-02-13T06:58:37.954 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: set os type and version on server
2022-02-13T06:58:37.955 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com'
2022-02-13T06:58:38.151 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64
2022-02-13T06:58:38.210 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream"
2022-02-13T06:58:38.210 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="8"
2022-02-13T06:58:38.210 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos"
2022-02-13T06:58:38.210 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora"
2022-02-13T06:58:38.211 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="8"
2022-02-13T06:58:38.211 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el8"
2022-02-13T06:58:38.211 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 8"
2022-02-13T06:58:38.211 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31"
2022-02-13T06:58:38.212 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-02-13T06:58:38.212 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/"
2022-02-13T06:58:38.212 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-02-13T06:58:38.212 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-02-13T06:58:38.212 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-02-13T06:58:38.213 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server
2022-02-13T06:58:38.322 INFO:teuthology.dispatcher.supervisor:Running job 6679197
2022-02-13T06:58:38.323 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-02-13_03:31:02-rados-pacific-distro-default-smithi/6679197 --name teuthology-2022-02-13_03:31:02-rados-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/teuthology-2022-02-13_03:31:02-rados-pacific-distro-default-smithi/6679197/orig.config.yaml
2022-02-13T06:58:38.326 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-02-13_03:31:02-rados-pacific-distro-default-smithi/6679197
2022-02-13T06:58:38.327 INFO:teuthology.dispatcher.supervisor:Job PID: 28112
2022-02-13T06:58:38.327 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-13T13:28:49.246 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-13T13:28:49.449 INFO:teuthology.kill:Killing Pids: {28112}
2022-02-13T13:28:50.711 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-02-13T13:28:50.712 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-02-13T13:28:50.713 INFO:teuthology.misc:Compressing logs...
2022-02-13T13:28:50.714 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com'
2022-02-13T13:28:50.913 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com'
2022-02-13T13:28:50.963 INFO:teuthology.orchestra.run.smithi079.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-13T13:28:51.158 INFO:teuthology.orchestra.run.smithi100.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-02-13T13:28:51.212 INFO:teuthology.orchestra.run.smithi079.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-13T13:28:51.212 INFO:teuthology.orchestra.run.smithi079.stderr:tar: Error is not recoverable: exiting now
2022-02-13T13:28:51.254 INFO:teuthology.orchestra.run.smithi100.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-02-13T13:28:51.254 INFO:teuthology.orchestra.run.smithi100.stderr:tar: Error is not recoverable: exiting now
2022-02-13T13:28:51.256 INFO:teuthology.misc:Compressing logs...
2022-02-13T13:28:51.508 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping
2022-02-13T13:28:51.687 INFO:teuthology.misc:Compressing logs...
2022-02-13T13:29:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/45b15818-8c9b-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-02-13T13:29:17.640 INFO:teuthology.orchestra.run.smithi100.stderr:gzip: /var/log/ceph/45b15818-8c9b-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-02-13T13:29:29.179 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/45b15818-8c9b-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-02-13T13:29:49.581 INFO:teuthology.orchestra.run.smithi100.stderr:gzip: /var/log/ceph/45b15818-8c9b-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-02-13T13:29:54.703 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/45b15818-8c9b-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-02-13T13:30:06.783 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/45b15818-8c9b-11ec-8c35-001a4aab830c/ceph-mon.smithi079.log: file size changed while zipping
2022-02-13T13:30:10.849 INFO:teuthology.orchestra.run.smithi100.stderr:gzip: /var/log/ceph/45b15818-8c9b-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-02-13T13:30:11.698 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/45b15818-8c9b-11ec-8c35-001a4aab830c/ceph-mgr.smithi079.zzxwsq.log: file size changed while zipping
2022-02-13T13:30:14.282 INFO:teuthology.orchestra.run.smithi100.stderr:gzip: /var/log/ceph/45b15818-8c9b-11ec-8c35-001a4aab830c/ceph-mon.smithi100.log: file size changed while zipping
2022-02-13T13:30:36.083 INFO:teuthology.orchestra.run.smithi100.stderr:gzip: /var/log/ceph/45b15818-8c9b-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-02-13T13:30:37.336 INFO:teuthology.orchestra.run.smithi079.stderr:gzip: /var/log/ceph/45b15818-8c9b-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-02-13T13:31:20.823 INFO:teuthology.kill:No teuthology processes running
2022-02-13T13:31:22.042 INFO:teuthology.kill:Nuking machines: ['smithi079', 'smithi100']
2022-02-13T13:31:23.104 INFO:teuthology.kill:2022-02-13 13:31:23,104.104 INFO:teuthology.nuke:targets:
2022-02-13T13:31:23.105 INFO:teuthology.kill:  smithi079.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC54YZAw6HO1MXIDoL6J32NrJnoxTL4J8I05LkiFwANqCkXrpui9S+Ul1dr5n7tLWO4aC41CfMRMEGrRAM9t3pri0wsxmKIulWFjy6Vt21RJ/jfAaB082D2w8iAiukzFU05kdqFRp2P/YTeaS7r9j7zHcCKFpuYBxbnLiR81FJwrXODkBYP5GmMnrr5n30/v9obGh471gCAR0YFLWsi7Uf0MGm8PUaRZIOev0iQCwF6k8S58g4oxW6jCUMddLaPaap6GYuZHVHBiTOJqyDR5APSnbuZPxBNeh7+cr9ctHPUB39V6eKczOmQZgCxiH+8/91MJE7XHCOy/BjkmelGKKHxVaa5NgKF9d6VH9LfCFv10YLavfejCmbTu7XCImwxnmfamjxwO07FVr6hwXWwUb1/+PgDPRxMdZDSICuuddl+TwyzuOHrHhruvyfhSalDL98wHdb1/iL0k+YUzGJEiRJqd8LYe1hBmfbjqG8EwABqTpI4yOvgs757OyV3SWaiATE=
2022-02-13T13:31:23.105 INFO:teuthology.kill:  smithi100.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOSz/ngvS79ekDELiUriBczaH4UmTrYfKoX9NIVji20sTdvMR3vjDLakkk+CapfZNpUbMNgt3z/x06rjQ+CBpSkMMQm7iR4kSR2Gi/UURTvdaHKOXpB7sP+ZcHcB4MdiETCWKw7lE5elOtBIdibW/NNvw1Kak13JMVe/GkuxFqn3xzOdaNDDrWidDCVot1/+vddLs8gfVf3bFsSrbSVJxCPQWTb6v1DciwRkVOLMa4IhLvIa2vQM6Tnh2pmY16/cKqTXQqfiVD6CNXyvmUZeuVx9GyIo3NqZ5PBkX5Vbb0SkkuQco37CcV0lkM9UGsTku0TzidJyvrBefgdXVcT0a7okjdz8ivBoFjoqG45FxAhdy+In7TTH5KGHJYfwAvFtEepkSw/nbshPyDpVG0E/oLszWSDZ9r7Lp1lgjK/y3ZRjD6LTofm2jeGd77QF9VgRuAESa9h+mOpGc8BeE3sI/5byMVtdr+b/g2lwjHlr/rUe1pUMrn4U1+Z2k8PQaCr6c=
2022-02-13T13:31:23.136 INFO:teuthology.kill:2022-02-13 13:31:23,135.135 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-13T13:31:23.140 INFO:teuthology.kill:2022-02-13 13:31:23,140.140 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-13T13:31:23.225 INFO:teuthology.kill:2022-02-13 13:31:23,225.225 INFO:teuthology.orchestra.console:Power off smithi100
2022-02-13T13:31:23.247 INFO:teuthology.kill:2022-02-13 13:31:23,246.246 INFO:teuthology.orchestra.console:Power off smithi079
2022-02-13T13:31:31.625 INFO:teuthology.kill:2022-02-13 13:31:31,624.624 INFO:teuthology.orchestra.console:Power off for smithi079 completed
2022-02-13T13:31:31.787 INFO:teuthology.kill:2022-02-13 13:31:31,786.786 INFO:teuthology.lock.ops:unlocked: smithi079.front.sepia.ceph.com
2022-02-13T13:31:35.731 INFO:teuthology.kill:2022-02-13 13:31:35,730.730 INFO:teuthology.orchestra.console:Power off for smithi100 completed
2022-02-13T13:31:35.877 INFO:teuthology.kill:2022-02-13 13:31:35,877.877 INFO:teuthology.lock.ops:unlocked: smithi100.front.sepia.ceph.com
2022-02-13T13:33:36.121 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-13T13:33:36.222 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi079 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-02-13_07:01:03-rados-master-distro-default-smithi/6679686
2022-02-13T13:33:36.223 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi100 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-02-13_07:01:03-rados-master-distro-default-smithi/6679688