2022-01-23T12:27:35.901 INFO:root:teuthology version: 1.1.0-75f1bcd2
2022-01-23T12:27:36.041 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi055.front.sepia.ceph.com' reimaging
2022-01-23T12:27:36.042 INFO:teuthology.lock.ops:Updating [ubuntu@smithi055.front.sepia.ceph.com]: reset os type and version on server
2022-01-23T12:27:36.042 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server
2022-01-23T12:27:36.065 INFO:teuthology.lock.ops:Node 'ubuntu@smithi055.front.sepia.ceph.com' reimaging is complete
2022-01-23T12:27:36.065 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi148.front.sepia.ceph.com' reimaging
2022-01-23T12:27:36.066 INFO:teuthology.lock.ops:Updating [ubuntu@smithi148.front.sepia.ceph.com]: reset os type and version on server
2022-01-23T12:27:36.067 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server
2022-01-23T12:27:36.087 INFO:teuthology.lock.ops:Node 'ubuntu@smithi148.front.sepia.ceph.com' reimaging is complete
2022-01-23T12:27:36.297 INFO:teuthology.provision.fog.smithi055:Scheduling deploy of centos 8.3
2022-01-23T12:27:36.324 INFO:teuthology.provision.fog.smithi148:Scheduling deploy of centos 8.3
2022-01-23T12:27:36.696 INFO:teuthology.orchestra.console:Power off smithi055
2022-01-23T12:27:36.816 INFO:teuthology.orchestra.console:Power off smithi148
2022-01-23T12:27:45.181 INFO:teuthology.orchestra.console:Power off for smithi148 completed
2022-01-23T12:27:45.283 INFO:teuthology.orchestra.console:Power on smithi148
2022-01-23T12:27:49.184 INFO:teuthology.orchestra.console:Power off for smithi055 completed
2022-01-23T12:27:49.286 INFO:teuthology.orchestra.console:Power on smithi055
2022-01-23T12:27:53.714 INFO:teuthology.orchestra.console:Power on for smithi148 completed
2022-01-23T12:27:53.815 INFO:teuthology.provision.fog.smithi148:Waiting for deploy to finish
2022-01-23T12:27:57.649 INFO:teuthology.orchestra.console:Power on for smithi055 completed
2022-01-23T12:27:57.750 INFO:teuthology.provision.fog.smithi055:Waiting for deploy to finish
2022-01-23T12:32:41.161 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-23T12:32:41.439 INFO:teuthology.provision.fog.smithi148:Node is ready
2022-01-23T12:32:41.454 INFO:teuthology.orchestra.run.smithi148.stdout:smithi148.front.sepia.ceph.com
2022-01-23T12:32:41.508 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.15.148 smithi148.front.sepia.ceph.com smithi148
2022-01-23T12:32:41.849 INFO:teuthology.provision.fog.smithi148:Deploy complete!
2022-01-23T12:32:50.871 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-01-23T12:32:51.184 INFO:teuthology.provision.fog.smithi055:Node is ready
2022-01-23T12:32:51.200 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055.front.sepia.ceph.com
2022-01-23T12:32:51.256 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55 smithi055.front.sepia.ceph.com smithi055
2022-01-23T12:32:51.588 INFO:teuthology.provision.fog.smithi055:Deploy complete!
2022-01-23T12:32:51.739 INFO:teuthology.lock.ops:Checking smithi055.front.sepia.ceph.com
2022-01-23T12:32:51.741 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-23T12:32:51.759 INFO:teuthology.lock.ops:Checking smithi148.front.sepia.ceph.com
2022-01-23T12:32:51.759 INFO:teuthology.lock.ops:New key found. Updating...
2022-01-23T12:32:51.777 INFO:teuthology.lock.ops:Updating [smithi055.front.sepia.ceph.com]: set os type and version on server
2022-01-23T12:32:51.777 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com'
2022-01-23T12:32:51.981 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64
2022-01-23T12:32:52.036 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="CentOS Linux"
2022-01-23T12:32:52.037 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="8"
2022-01-23T12:32:52.037 INFO:teuthology.orchestra.run.smithi055.stdout:ID="centos"
2022-01-23T12:32:52.037 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="rhel fedora"
2022-01-23T12:32:52.038 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="8"
2022-01-23T12:32:52.038 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el8"
2022-01-23T12:32:52.038 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="CentOS Linux 8"
2022-01-23T12:32:52.039 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31"
2022-01-23T12:32:52.039 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-01-23T12:32:52.039 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://centos.org/"
2022-01-23T12:32:52.039 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2022-01-23T12:32:52.039 INFO:teuthology.orchestra.run.smithi055.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2022-01-23T12:32:52.040 INFO:teuthology.orchestra.run.smithi055.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2022-01-23T12:32:52.041 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server
2022-01-23T12:32:52.059 INFO:teuthology.lock.ops:Updating [smithi148.front.sepia.ceph.com]: set os type and version on server
2022-01-23T12:32:52.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com'
2022-01-23T12:32:52.324 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64
2022-01-23T12:32:52.378 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Linux"
2022-01-23T12:32:52.379 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="8"
2022-01-23T12:32:52.379 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos"
2022-01-23T12:32:52.379 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora"
2022-01-23T12:32:52.379 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="8"
2022-01-23T12:32:52.380 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el8"
2022-01-23T12:32:52.380 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Linux 8"
2022-01-23T12:32:52.380 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31"
2022-01-23T12:32:52.380 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-01-23T12:32:52.380 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/"
2022-01-23T12:32:52.381 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2022-01-23T12:32:52.381 INFO:teuthology.orchestra.run.smithi148.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2022-01-23T12:32:52.381 INFO:teuthology.orchestra.run.smithi148.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2022-01-23T12:32:52.382 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server
2022-01-23T12:32:52.476 INFO:teuthology.dispatcher.supervisor:Running job 6635656
2022-01-23T12:32:52.477 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_75f1bcd237a8b6022305cd53ba87e106d45aa665/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-01-23_03:31:03-rados-pacific-distro-default-smithi/6635656 --name teuthology-2022-01-23_03:31:03-rados-pacific-distro-default-smithi --description rados/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.3_container_tools_3.0 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/teuthology-2022-01-23_03:31:03-rados-pacific-distro-default-smithi/6635656/orig.config.yaml
2022-01-23T12:32:52.480 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-01-23_03:31:03-rados-pacific-distro-default-smithi/6635656
2022-01-23T12:32:52.481 INFO:teuthology.dispatcher.supervisor:Job PID: 29898
2022-01-23T12:32:52.481 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-01-23T19:03:21.491 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-01-23T19:03:21.670 INFO:teuthology.kill:Killing Pids: {29898}
2022-01-23T19:03:22.855 INFO:teuthology.task.internal:roles: ubuntu@smithi055.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-01-23T19:03:22.856 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-01-23T19:03:22.857 INFO:teuthology.misc:Compressing logs...
2022-01-23T19:03:22.858 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com'
2022-01-23T19:03:23.075 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com'
2022-01-23T19:03:23.124 INFO:teuthology.orchestra.run.smithi055.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-01-23T19:03:23.322 INFO:teuthology.orchestra.run.smithi148.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-01-23T19:03:23.369 INFO:teuthology.orchestra.run.smithi055.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-01-23T19:03:23.370 INFO:teuthology.orchestra.run.smithi055.stderr:tar: Error is not recoverable: exiting now
2022-01-23T19:03:23.411 INFO:teuthology.orchestra.run.smithi148.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-01-23T19:03:23.411 INFO:teuthology.orchestra.run.smithi148.stderr:tar: Error is not recoverable: exiting now
2022-01-23T19:03:23.412 INFO:teuthology.misc:Compressing logs...
2022-01-23T19:03:23.669 INFO:teuthology.misc:Compressing logs...
2022-01-23T19:03:48.104 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/dd604622-7c49-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-01-23T19:03:48.975 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: /var/log/ceph/dd604622-7c49-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-01-23T19:03:52.903 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/dd604622-7c49-11ec-8c35-001a4aab830c/ceph-mgr.smithi055.nuwikp.log: file size changed while zipping
2022-01-23T19:04:04.845 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/dd604622-7c49-11ec-8c35-001a4aab830c/ceph-mon.smithi055.log: file size changed while zipping
2022-01-23T19:04:10.320 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: /var/log/ceph/dd604622-7c49-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-01-23T19:04:25.677 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/dd604622-7c49-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-01-23T19:04:29.557 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: /var/log/ceph/dd604622-7c49-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-01-23T19:04:32.952 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: /var/log/ceph/dd604622-7c49-11ec-8c35-001a4aab830c/ceph-mon.smithi148.log: file size changed while zipping
2022-01-23T19:04:40.273 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/dd604622-7c49-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-01-23T19:04:51.053 INFO:teuthology.orchestra.run.smithi148.stderr:gzip: /var/log/ceph/dd604622-7c49-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-01-23T19:05:03.725 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/dd604622-7c49-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-01-23T19:05:42.039 INFO:teuthology.kill:No teuthology processes running
2022-01-23T19:05:43.164 INFO:teuthology.kill:Nuking machines: ['smithi055', 'smithi148']
2022-01-23T19:05:44.143 INFO:teuthology.kill:2022-01-23 19:05:44,142.142 INFO:teuthology.nuke:targets:
2022-01-23T19:05:44.144 INFO:teuthology.kill:  smithi055.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAW9ktTMM4QL8Rj3D5Bo84bqrGDHKp/UbtxVZw8kaHNcv2W1ZgSLX9apWUPcotbjaoWn3/SBgzQmo/1RHWqJt6OO5qkUykmHOxnv8kiPDlS4fZqEsyEL4lClN+o5wq0yMrHiHxGPeDYsX7SglNOSz1NEU5LmLcugy71vDRIGclFD99xftISXW4eXLt118lq4uMWD6sjhZ7W/+cJ2aNqIj0Na3MXaMxOkCloUU4zNvEQevf1Sp5pqeRbvog9aweR+2dxWdmzMbAurHFs6mujYKauESoR7fpAVaUYv3fJUfUFWn9GbV0qRB0dRITjgCQiVzTmsXci4NqzuzSN8/ypcbJQPybCZLLXiu4/bxvB+BZ0FQGyt7WCbsfqVOyD+yYe23s0SFU6QX1eQgnGiyTBHb9wefieQr9RSRJ52EFHwNZO9Z1S2Op8eiKXXBTfQVEnmkpDAQFgnaBdnVhevHWzKnNbTAR3Itd7ubfHNs4jBFXwSoYFYnjHCQXSpWqzHwJsRc=
2022-01-23T19:05:44.144 INFO:teuthology.kill:  smithi148.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1xaVOPsJ1i6P+s5dTR9YiVbaOypp+/lpNrVX2O3I9WVUTzOZHSH2veWUnpqFdju77UMpiKOQg0x6tvAnqpUfW1G2SMEfDUtyvHQenSuyoowm7fT5Y0PxEVMrEmLd5E0bZ4RiYkknWDNYD5zlAHs6DkHkMcWSmcmcndqUXve7XPetcppMWrG3Lmg8FFKnReqOi9AKNS+P5tVE+mJwTX8qqjpsT0kjMkjNkKQ7JOARwOJc6g7Lg5O21Js95RK7lSoO+49naQo1bD57Oc7GhRODz7ssiTz1UYr0k0fXvYT+SuZWYOFpSqY8Msw+dm6uXKFXckppIYhm52Ug0XdQ7XKFengbdOI6qa7cbHAf8t/782PP/UVuJYr3Z6vCdQb7nI+d/c207wJpcIxD6H1+F7ZtkHb9YUQaTG5kYPzskL71RPKUWXAkXkMBmbdzwSzyC/3DoU2uOkA/A8saOr+W8GiFB4KUzaE8dglfGJ9oflOIOfZ1vqHo49H/ZSofkKIRnvWM=
2022-01-23T19:05:44.169 INFO:teuthology.kill:2022-01-23 19:05:44,169.169 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-23T19:05:44.172 INFO:teuthology.kill:2022-01-23 19:05:44,172.172 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-01-23T19:05:44.246 INFO:teuthology.kill:2022-01-23 19:05:44,246.246 INFO:teuthology.orchestra.console:Power off smithi055
2022-01-23T19:05:44.259 INFO:teuthology.kill:2022-01-23 19:05:44,258.258 INFO:teuthology.orchestra.console:Power off smithi148
2022-01-23T19:05:52.624 INFO:teuthology.kill:2022-01-23 19:05:52,623.623 INFO:teuthology.orchestra.console:Power off for smithi148 completed
2022-01-23T19:05:52.770 INFO:teuthology.kill:2022-01-23 19:05:52,769.769 INFO:teuthology.lock.ops:unlocked: smithi148.front.sepia.ceph.com
2022-01-23T19:05:56.733 INFO:teuthology.kill:2022-01-23 19:05:56,733.733 INFO:teuthology.orchestra.console:Power off for smithi055 completed
2022-01-23T19:05:56.887 INFO:teuthology.kill:2022-01-23 19:05:56,887.887 INFO:teuthology.lock.ops:unlocked: smithi055.front.sepia.ceph.com
2022-01-23T19:07:57.136 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-01-23T19:07:57.229 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi055 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-20_03:05:02-rgw-master-distro-default-smithi/6628331
2022-01-23T19:07:57.229 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi148 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-01-20_03:05:02-rgw-master-distro-default-smithi/6628331