2022-03-21T22:23:17.118 INFO:root:teuthology version: 1.1.0-a4e18662
2022-03-21T22:23:17.240 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi025.front.sepia.ceph.com' reimaging
2022-03-21T22:23:17.241 INFO:teuthology.lock.ops:Updating [ubuntu@smithi025.front.sepia.ceph.com]: reset os type and version on server
2022-03-21T22:23:17.242 INFO:teuthology.lock.ops:Updating smithi025.front.sepia.ceph.com on lock server
2022-03-21T22:23:17.263 INFO:teuthology.lock.ops:Node 'ubuntu@smithi025.front.sepia.ceph.com' reimaging is complete
2022-03-21T22:23:17.263 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi027.front.sepia.ceph.com' reimaging
2022-03-21T22:23:17.264 INFO:teuthology.lock.ops:Updating [ubuntu@smithi027.front.sepia.ceph.com]: reset os type and version on server
2022-03-21T22:23:17.264 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server
2022-03-21T22:23:17.282 INFO:teuthology.lock.ops:Node 'ubuntu@smithi027.front.sepia.ceph.com' reimaging is complete
2022-03-21T22:23:17.477 INFO:teuthology.provision.fog.smithi025:Scheduling deploy of centos 8.stream
2022-03-21T22:23:17.488 INFO:teuthology.provision.fog.smithi027:Scheduling deploy of centos 8.stream
2022-03-21T22:23:17.830 INFO:teuthology.orchestra.console:Power off smithi025
2022-03-21T22:23:17.843 INFO:teuthology.orchestra.console:Power off smithi027
2022-03-21T22:23:17.973 INFO:teuthology.orchestra.console:Power off for smithi025 completed
2022-03-21T22:23:17.985 INFO:teuthology.orchestra.console:Power off for smithi027 completed
2022-03-21T22:23:18.075 INFO:teuthology.orchestra.console:Power on smithi025
2022-03-21T22:23:18.086 INFO:teuthology.orchestra.console:Power on smithi027
2022-03-21T22:23:26.442 INFO:teuthology.orchestra.console:Power on for smithi025 completed
2022-03-21T22:23:26.448 INFO:teuthology.orchestra.console:Power on for smithi027 completed
2022-03-21T22:23:26.544 INFO:teuthology.provision.fog.smithi025:Waiting for deploy to finish
2022-03-21T22:23:26.550 INFO:teuthology.provision.fog.smithi027:Waiting for deploy to finish
2022-03-21T22:28:12.160 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-21T22:28:32.447 INFO:teuthology.provision.fog.smithi027:Node is ready
2022-03-21T22:28:32.470 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027.front.sepia.ceph.com
2022-03-21T22:28:32.530 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27 smithi027.front.sepia.ceph.com smithi027
2022-03-21T22:28:32.903 INFO:teuthology.provision.fog.smithi027:Deploy complete!
2022-03-21T22:28:43.284 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-21T22:28:43.646 INFO:teuthology.provision.fog.smithi025:Node is ready
2022-03-21T22:28:43.665 INFO:teuthology.orchestra.run.smithi025.stdout:smithi025.front.sepia.ceph.com
2022-03-21T22:28:43.724 INFO:teuthology.orchestra.run.smithi025.stdout:172.21.15.25 smithi025.front.sepia.ceph.com smithi025
2022-03-21T22:28:44.089 INFO:teuthology.provision.fog.smithi025:Deploy complete!
2022-03-21T22:28:44.282 INFO:teuthology.lock.ops:Checking smithi025.front.sepia.ceph.com
2022-03-21T22:28:44.283 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-21T22:28:44.303 INFO:teuthology.lock.ops:Checking smithi027.front.sepia.ceph.com
2022-03-21T22:28:44.303 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-21T22:28:44.320 INFO:teuthology.lock.ops:Updating [smithi025.front.sepia.ceph.com]: set os type and version on server
2022-03-21T22:28:44.321 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi025.front.sepia.ceph.com'
2022-03-21T22:28:44.512 INFO:teuthology.orchestra.run.smithi025.stdout:x86_64
2022-03-21T22:28:44.569 INFO:teuthology.orchestra.run.smithi025.stdout:NAME="CentOS Stream"
2022-03-21T22:28:44.569 INFO:teuthology.orchestra.run.smithi025.stdout:VERSION="8"
2022-03-21T22:28:44.569 INFO:teuthology.orchestra.run.smithi025.stdout:ID="centos"
2022-03-21T22:28:44.570 INFO:teuthology.orchestra.run.smithi025.stdout:ID_LIKE="rhel fedora"
2022-03-21T22:28:44.570 INFO:teuthology.orchestra.run.smithi025.stdout:VERSION_ID="8"
2022-03-21T22:28:44.570 INFO:teuthology.orchestra.run.smithi025.stdout:PLATFORM_ID="platform:el8"
2022-03-21T22:28:44.570 INFO:teuthology.orchestra.run.smithi025.stdout:PRETTY_NAME="CentOS Stream 8"
2022-03-21T22:28:44.571 INFO:teuthology.orchestra.run.smithi025.stdout:ANSI_COLOR="0;31"
2022-03-21T22:28:44.571 INFO:teuthology.orchestra.run.smithi025.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-03-21T22:28:44.571 INFO:teuthology.orchestra.run.smithi025.stdout:HOME_URL="https://centos.org/"
2022-03-21T22:28:44.571 INFO:teuthology.orchestra.run.smithi025.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-21T22:28:44.572 INFO:teuthology.orchestra.run.smithi025.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-21T22:28:44.572 INFO:teuthology.orchestra.run.smithi025.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-03-21T22:28:44.573 INFO:teuthology.lock.ops:Updating smithi025.front.sepia.ceph.com on lock server
2022-03-21T22:28:44.591 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: set os type and version on server
2022-03-21T22:28:44.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com'
2022-03-21T22:28:44.842 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64
2022-03-21T22:28:44.900 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream"
2022-03-21T22:28:44.900 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="8"
2022-03-21T22:28:44.900 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos"
2022-03-21T22:28:44.901 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora"
2022-03-21T22:28:44.901 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="8"
2022-03-21T22:28:44.901 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el8"
2022-03-21T22:28:44.901 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 8"
2022-03-21T22:28:44.901 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31"
2022-03-21T22:28:44.902 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-03-21T22:28:44.902 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/"
2022-03-21T22:28:44.902 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-03-21T22:28:44.902 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-03-21T22:28:44.903 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-03-21T22:28:44.903 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server
2022-03-21T22:28:45.004 INFO:teuthology.dispatcher.supervisor:Running job 6750590
2022-03-21T22:28:45.005 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-03-21_21:15:51-orch-wip-yuri10-testing-2022-03-16-1432-pacific-distro-default-smithi/6750590 --name yuriw-2022-03-21_21:15:51-orch-wip-yuri10-testing-2022-03-16-1432-pacific-distro-default-smithi --description orch/cephadm/mgr-nfs-upgrade/{0-distro/centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2022-03-21_21:15:51-orch-wip-yuri10-testing-2022-03-16-1432-pacific-distro-default-smithi/6750590/orig.config.yaml
2022-03-21T22:28:45.008 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-03-21_21:15:51-orch-wip-yuri10-testing-2022-03-16-1432-pacific-distro-default-smithi/6750590
2022-03-21T22:28:45.009 INFO:teuthology.dispatcher.supervisor:Job PID: 24459
2022-03-21T22:28:45.009 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-03-21T22:29:46.063 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2022-03-22T04:59:14.086 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-03-22T04:59:14.246 INFO:teuthology.kill:Killing Pids: {24459}
2022-03-22T04:59:15.488 INFO:teuthology.task.internal:roles: ubuntu@smithi025.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-03-22T04:59:15.489 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-03-22T04:59:15.490 INFO:teuthology.misc:Compressing logs...
2022-03-22T04:59:15.491 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi025.front.sepia.ceph.com'
2022-03-22T04:59:15.689 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com'
2022-03-22T04:59:15.740 INFO:teuthology.orchestra.run.smithi025.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-03-22T04:59:15.932 INFO:teuthology.orchestra.run.smithi027.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-03-22T04:59:15.979 INFO:teuthology.orchestra.run.smithi025.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-03-22T04:59:15.980 INFO:teuthology.orchestra.run.smithi025.stderr:tar: Error is not recoverable: exiting now
2022-03-22T04:59:16.020 INFO:teuthology.orchestra.run.smithi027.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-03-22T04:59:16.020 INFO:teuthology.orchestra.run.smithi027.stderr:tar: Error is not recoverable: exiting now
2022-03-22T04:59:16.022 INFO:teuthology.misc:Compressing logs...
2022-03-22T04:59:16.329 INFO:teuthology.misc:Compressing logs...
2022-03-22T04:59:18.488 INFO:teuthology.orchestra.run.smithi027.stderr:gzip: /var/log/ceph/8453f3da-a967-11ec-8c35-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-03-22T04:59:19.409 INFO:teuthology.orchestra.run.smithi025.stderr:gzip: /var/log/ceph/8453f3da-a967-11ec-8c35-001a4aab830c/ceph-mon.smithi025.log: file size changed while zipping
2022-03-22T04:59:20.863 INFO:teuthology.orchestra.run.smithi027.stderr:gzip: /var/log/ceph/8453f3da-a967-11ec-8c35-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-03-22T04:59:21.595 INFO:teuthology.orchestra.run.smithi027.stderr:gzip: /var/log/ceph/8453f3da-a967-11ec-8c35-001a4aab830c/ceph-mon.smithi027.log: file size changed while zipping
2022-03-22T04:59:21.827 INFO:teuthology.orchestra.run.smithi025.stderr:gzip: /var/log/ceph/8453f3da-a967-11ec-8c35-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-03-22T04:59:23.566 INFO:teuthology.orchestra.run.smithi027.stderr:gzip: /var/log/ceph/8453f3da-a967-11ec-8c35-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-03-22T04:59:23.811 INFO:teuthology.orchestra.run.smithi025.stderr:gzip: /var/log/ceph/8453f3da-a967-11ec-8c35-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-03-22T04:59:24.462 INFO:teuthology.orchestra.run.smithi025.stderr:gzip: /var/log/ceph/8453f3da-a967-11ec-8c35-001a4aab830c/ceph-mgr.smithi025.cqgzpj.log: file size changed while zipping
2022-03-22T04:59:25.980 INFO:teuthology.orchestra.run.smithi027.stderr:gzip: /var/log/ceph/8453f3da-a967-11ec-8c35-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-03-22T04:59:27.396 INFO:teuthology.orchestra.run.smithi025.stderr:gzip: /var/log/ceph/8453f3da-a967-11ec-8c35-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-03-22T04:59:30.551 INFO:teuthology.orchestra.run.smithi025.stderr:gzip: /var/log/ceph/8453f3da-a967-11ec-8c35-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-03-22T05:00:05.486 INFO:teuthology.kill:No teuthology processes running
2022-03-22T05:00:06.626 INFO:teuthology.kill:Nuking machines: ['smithi025', 'smithi027']
2022-03-22T05:00:07.653 INFO:teuthology.kill:2022-03-22 05:00:07,653.653 INFO:teuthology.nuke:targets:
2022-03-22T05:00:07.654 INFO:teuthology.kill:  smithi025.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8SXhfnINaG9SwB1NwINMI3EA65cdGSr1RGgnEZjYEy79VxgyHMj0L3VNsKWkeD3BJeB5v5cXB8jY2RGM9DFOrN43at+oPxnaDmegCav2vO9JnbHjTRVucCiL/1M+05SyG6eq+6F1TMonla0TDDfi2f8qkqsZtieGpq8MISw6tl0zn9TO2B/Bi1AISd690/gQnoZH4+SZCEz9wlpxeuUML1RQJuyXe9cLhDw1HKL8Nrbx4ReHEe+x66lY7/xzizEA+U6dZQLiGERpONjWnGQ0zawitWc/DIHj2NseafZBRBFPJnKK276cAZsehfXemjGHVi+jA+3p1SXNmaw/63gkKEP0Jv8sXKhwnyGPiHBAzP2iMfU/J5rGeLo8owD+Ac0myPISgLCJWDoW7a+/Y5oZ7K+f3BW2Pb8SqE0S7lU+YLltvQt4d+FTHlWLQ1Xx60GaOm9CLXMQqhJ/08B/ZV1AfO1qRaUchUqubj5Xg+lsN7Sg8v4mlT1njh+DsulgH7X0=
2022-03-22T05:00:07.654 INFO:teuthology.kill:  smithi027.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDH1G8PpYPQqdlYsp1G0VjW8a7MAkjGnpyjZoxu5ThfZDc6hR30KXY8KBH2+T5utKWlfFaIriuiDjKqPc6HG10oi5vCAqUNyORGwnWyJcuTPhzu0a5GWOD9yArHPwfzymRIgP0YpGMyR0vS8N3c4kOo6etkMN52a0qsM+WKQHArUYLIN8/EjH0KyeGdzH45DRr0Lu5ggWDaXi+k61JPOE4fpinvTXD9IA46TtFUMqWBkyFmzbcY58QC0z3QumELUzftZDznKJGp9yP4CUh5+7uO5/wC4NIU9cfrrm7yNo3zqVEUP2lmIP70zJym5AZcD169Nxu/FwMlqNr4nzlKBPQmitiW0duCh2vtvjqXIWy8DKQwkNVvqlFRigUVZPasA/VaSaHXXZjG8qy/45R2SCuCLQJ3BI/06oZz5rpw2SH4fffT6tGlENTAg77F3rO0APj10DcMulKpZ/jI5PYIVHSVpkmRGtlKxIxoULLvl2pTE3rJnyva/WaBUZY3COviVGE=
2022-03-22T05:00:07.683 INFO:teuthology.kill:2022-03-22 05:00:07,683.683 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-22T05:00:07.687 INFO:teuthology.kill:2022-03-22 05:00:07,687.687 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-22T05:00:07.766 INFO:teuthology.kill:2022-03-22 05:00:07,766.766 INFO:teuthology.orchestra.console:Power off smithi027
2022-03-22T05:00:07.783 INFO:teuthology.kill:2022-03-22 05:00:07,783.783 INFO:teuthology.orchestra.console:Power off smithi025
2022-03-22T05:00:16.145 INFO:teuthology.kill:2022-03-22 05:00:16,144.144 INFO:teuthology.orchestra.console:Power off for smithi027 completed
2022-03-22T05:00:16.291 INFO:teuthology.kill:2022-03-22 05:00:16,290.290 INFO:teuthology.lock.ops:unlocked: smithi027.front.sepia.ceph.com
2022-03-22T05:00:20.264 INFO:teuthology.kill:2022-03-22 05:00:20,263.263 INFO:teuthology.orchestra.console:Power off for smithi025 completed
2022-03-22T05:00:20.408 INFO:teuthology.kill:2022-03-22 05:00:20,408.408 INFO:teuthology.lock.ops:unlocked: smithi025.front.sepia.ceph.com
2022-03-22T05:02:20.706 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-03-22T05:02:20.796 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi027 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-03-19_14:23:02-upgrade:nautilus-x-pacific-distro-default-smithi/6746535