2022-12-03T02:29:16.527 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf
2022-12-03T02:29:16.670 INFO:teuthology.lock.ops:Start node 'smithi174.front.sepia.ceph.com' reimaging
2022-12-03T02:29:16.671 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: reset os type and version on server
2022-12-03T02:29:16.672 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server
2022-12-03T02:29:16.693 INFO:teuthology.lock.ops:Node 'smithi174.front.sepia.ceph.com' reimaging is complete
2022-12-03T02:29:16.694 INFO:teuthology.lock.ops:Start node 'smithi182.front.sepia.ceph.com' reimaging
2022-12-03T02:29:16.695 INFO:teuthology.lock.ops:Updating [smithi182.front.sepia.ceph.com]: reset os type and version on server
2022-12-03T02:29:16.695 INFO:teuthology.lock.ops:Updating smithi182.front.sepia.ceph.com on lock server
2022-12-03T02:29:16.721 INFO:teuthology.lock.ops:Node 'smithi182.front.sepia.ceph.com' reimaging is complete
2022-12-03T02:29:16.911 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of centos 8.stream
2022-12-03T02:29:16.936 INFO:teuthology.provision.fog.smithi182:Scheduling deploy of centos 8.stream
2022-12-03T02:29:17.266 INFO:teuthology.orchestra.console:Power off smithi174
2022-12-03T02:29:17.282 INFO:teuthology.orchestra.console:Power off smithi182
2022-12-03T02:29:25.638 INFO:teuthology.orchestra.console:Power off for smithi174 completed
2022-12-03T02:29:25.739 INFO:teuthology.orchestra.console:Power on smithi174
2022-12-03T02:29:29.761 INFO:teuthology.orchestra.console:Power off for smithi182 completed
2022-12-03T02:29:29.863 INFO:teuthology.orchestra.console:Power on smithi182
2022-12-03T02:29:34.112 INFO:teuthology.orchestra.console:Power on for smithi174 completed
2022-12-03T02:29:34.122 INFO:teuthology.orchestra.console:Power on for smithi182 completed
2022-12-03T02:29:34.215 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish
2022-12-03T02:29:34.223 INFO:teuthology.provision.fog.smithi182:Waiting for deploy to finish
2022-12-03T02:32:05.469 ERROR:teuthology.orchestra.connection:Error authenticating with smithi182.front.sepia.ceph.com: Authentication failed.
2022-12-03T02:33:37.866 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-12-03T02:34:03.197 INFO:teuthology.provision.fog.smithi174:Node is ready
2022-12-03T02:34:03.218 INFO:teuthology.orchestra.run.smithi174.stdout:smithi174.front.sepia.ceph.com
2022-12-03T02:34:03.277 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.174 smithi174.front.sepia.ceph.com smithi174
2022-12-03T02:34:03.632 INFO:teuthology.provision.fog.smithi174:Deploy complete!
2022-12-03T02:34:14.332 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-12-03T02:34:14.595 INFO:teuthology.provision.fog.smithi182:Node is ready
2022-12-03T02:34:14.613 INFO:teuthology.orchestra.run.smithi182.stdout:smithi182.front.sepia.ceph.com
2022-12-03T02:34:14.672 INFO:teuthology.orchestra.run.smithi182.stdout:172.21.15.182 smithi182.front.sepia.ceph.com smithi182
2022-12-03T02:34:15.022 INFO:teuthology.provision.fog.smithi182:Deploy complete!
2022-12-03T02:34:15.209 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com
2022-12-03T02:34:15.210 INFO:teuthology.lock.ops:New key found. Updating...
2022-12-03T02:34:15.233 INFO:teuthology.lock.ops:Checking smithi182.front.sepia.ceph.com
2022-12-03T02:34:15.233 INFO:teuthology.lock.ops:New key found. Updating...
2022-12-03T02:34:15.256 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server
2022-12-03T02:34:15.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2022-12-03T02:34:15.465 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64
2022-12-03T02:34:15.523 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream"
2022-12-03T02:34:15.523 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="8"
2022-12-03T02:34:15.524 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos"
2022-12-03T02:34:15.524 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora"
2022-12-03T02:34:15.524 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="8"
2022-12-03T02:34:15.524 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el8"
2022-12-03T02:34:15.525 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 8"
2022-12-03T02:34:15.525 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31"
2022-12-03T02:34:15.525 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-12-03T02:34:15.525 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/"
2022-12-03T02:34:15.526 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-12-03T02:34:15.526 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-12-03T02:34:15.526 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-12-03T02:34:15.528 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server
2022-12-03T02:34:15.549 INFO:teuthology.lock.ops:Updating [smithi182.front.sepia.ceph.com]: set os type and version on server
2022-12-03T02:34:15.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi182.front.sepia.ceph.com'
2022-12-03T02:34:15.802 INFO:teuthology.orchestra.run.smithi182.stdout:x86_64
2022-12-03T02:34:15.862 INFO:teuthology.orchestra.run.smithi182.stdout:NAME="CentOS Stream"
2022-12-03T02:34:15.862 INFO:teuthology.orchestra.run.smithi182.stdout:VERSION="8"
2022-12-03T02:34:15.862 INFO:teuthology.orchestra.run.smithi182.stdout:ID="centos"
2022-12-03T02:34:15.863 INFO:teuthology.orchestra.run.smithi182.stdout:ID_LIKE="rhel fedora"
2022-12-03T02:34:15.863 INFO:teuthology.orchestra.run.smithi182.stdout:VERSION_ID="8"
2022-12-03T02:34:15.863 INFO:teuthology.orchestra.run.smithi182.stdout:PLATFORM_ID="platform:el8"
2022-12-03T02:34:15.863 INFO:teuthology.orchestra.run.smithi182.stdout:PRETTY_NAME="CentOS Stream 8"
2022-12-03T02:34:15.864 INFO:teuthology.orchestra.run.smithi182.stdout:ANSI_COLOR="0;31"
2022-12-03T02:34:15.864 INFO:teuthology.orchestra.run.smithi182.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2022-12-03T02:34:15.864 INFO:teuthology.orchestra.run.smithi182.stdout:HOME_URL="https://centos.org/"
2022-12-03T02:34:15.865 INFO:teuthology.orchestra.run.smithi182.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2022-12-03T02:34:15.865 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2022-12-03T02:34:15.865 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2022-12-03T02:34:15.866 INFO:teuthology.lock.ops:Updating smithi182.front.sepia.ceph.com on lock server
2022-12-03T02:34:15.988 INFO:teuthology.dispatcher.supervisor:Running job 7101930
2022-12-03T02:34:15.989 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_4da97cf64e542f347ec47b7bdbe5eca99759f9b7/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2022-12-02_21:56:44-orch:cephadm-wip-adk-testing-2022-12-02-0932-distro-default-smithi/7101930 --name adking-2022-12-02_21:56:44-orch:cephadm-wip-adk-testing-2022-12-02-0932-distro-default-smithi --description orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/adking-2022-12-02_21:56:44-orch:cephadm-wip-adk-testing-2022-12-02-0932-distro-default-smithi/7101930/orig.config.yaml
2022-12-03T02:34:15.994 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2022-12-02_21:56:44-orch:cephadm-wip-adk-testing-2022-12-02-0932-distro-default-smithi/7101930
2022-12-03T02:34:15.995 INFO:teuthology.dispatcher.supervisor:Job PID: 20460
2022-12-03T02:34:15.995 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-12-03T14:34:40.463 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-12-03T14:34:40.666 INFO:teuthology.kill:Killing Pids: {20460}
2022-12-03T14:34:42.693 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2022-12-03T14:34:42.694 INFO:teuthology.task.internal:roles: ubuntu@smithi182.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2022-12-03T14:34:42.695 INFO:teuthology.misc:Compressing logs...
2022-12-03T14:34:42.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2022-12-03T14:34:42.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi182.front.sepia.ceph.com'
2022-12-03T14:34:42.944 INFO:teuthology.orchestra.run.smithi174.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-12-03T14:34:43.232 INFO:teuthology.orchestra.run.smithi182.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory
2022-12-03T14:34:43.279 INFO:teuthology.orchestra.run.smithi174.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-12-03T14:34:43.279 INFO:teuthology.orchestra.run.smithi174.stderr:tar: Error is not recoverable: exiting now
2022-12-03T14:34:43.318 INFO:teuthology.orchestra.run.smithi182.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory
2022-12-03T14:34:43.319 INFO:teuthology.orchestra.run.smithi182.stderr:tar: Error is not recoverable: exiting now
2022-12-03T14:34:43.322 INFO:teuthology.misc:Compressing logs...
2022-12-03T14:34:43.617 INFO:teuthology.misc:Compressing logs...
2022-12-03T14:34:50.488 INFO:teuthology.orchestra.run.smithi182.stderr:gzip: /var/log/ceph/1291c68e-72b4-11ed-843e-001a4aab830c/ceph-mon.smithi182.log: file size changed while zipping
2022-12-03T14:35:02.561 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/1291c68e-72b4-11ed-843e-001a4aab830c/ceph-mon.smithi174.log: file size changed while zipping
2022-12-03T14:35:15.123 INFO:teuthology.orchestra.run.smithi182.stderr:gzip: /var/log/ceph/1291c68e-72b4-11ed-843e-001a4aab830c/ceph-osd.4.log: file size changed while zipping
2022-12-03T14:35:32.886 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/1291c68e-72b4-11ed-843e-001a4aab830c/ceph-osd.3.log: file size changed while zipping
2022-12-03T14:35:36.296 INFO:teuthology.orchestra.run.smithi182.stderr:gzip: /var/log/ceph/1291c68e-72b4-11ed-843e-001a4aab830c/ceph-osd.6.log: file size changed while zipping
2022-12-03T14:36:04.384 INFO:teuthology.orchestra.run.smithi182.stderr:gzip: /var/log/ceph/1291c68e-72b4-11ed-843e-001a4aab830c/ceph-osd.7.log: file size changed while zipping
2022-12-03T14:36:04.407 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/1291c68e-72b4-11ed-843e-001a4aab830c/ceph-osd.1.log: file size changed while zipping
2022-12-03T14:36:28.775 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/1291c68e-72b4-11ed-843e-001a4aab830c/ceph-osd.0.log: file size changed while zipping
2022-12-03T14:36:29.993 INFO:teuthology.orchestra.run.smithi182.stderr:gzip: /var/log/ceph/1291c68e-72b4-11ed-843e-001a4aab830c/ceph-osd.5.log: file size changed while zipping
2022-12-03T14:36:47.287 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/1291c68e-72b4-11ed-843e-001a4aab830c/ceph-osd.2.log: file size changed while zipping
2022-12-03T14:36:53.124 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/1291c68e-72b4-11ed-843e-001a4aab830c/ceph-mgr.smithi174.onqxve.log: file size changed while zipping
2022-12-03T14:37:49.491 INFO:teuthology.kill:No teuthology processes running
2022-12-03T14:37:51.769 INFO:teuthology.kill:Nuking machines: ['smithi174', 'smithi182']
2022-12-03T14:37:53.392 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_teuthology_4da97cf64e542f347ec47b7bdbe5eca99759f9b7/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
2022-12-03T14:37:53.393 INFO:teuthology.kill:  from cryptography.hazmat.backends import default_backend
2022-12-03T14:37:53.394 INFO:teuthology.kill:2022-12-03 14:37:53,391.391 INFO:teuthology.nuke:targets:
2022-12-03T14:37:53.394 INFO:teuthology.kill:  smithi174.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4VG1GV/IPCV5Ap88cVDuRJeENf/e4SQ58WuGl5PjBPr4WzQm2omcX7Wiw/GbgQJsSWYjhD8FitA9fh8RMIpwB9jU8TFSyvfPicEwMOBwMq1yJfjG952+kyHUeC3IIx5eAzn8vzADwJevt8tQhUOm2sTyKTKcDA56sEtFGVmJwMGlMYfGjnqeqUgE9TSvaLmWOVv3t1C607VCiZ5K4N7bVNXA4jhYIGhsKVIH7feN9esjal/qIluI15ia9KJ9+/jFxnxcB3pmxIyawAcXTHwl/b/lCkhISAjNbiPOEHRuch9hg8zpqBKncSRMcTUqZGMyBgf/lD3WgrrPhwssIJpf0I0nE8T0Q1xJR+hb8Gh8yY6dl6+K5vq44V9HA8tamBj5DcjjBU1DqG7cms+P57rauAmaq+TnYP4i7Hvrz5qz7aYmw2/5+s27oVZFikQ9ZlP0mKBTAl9PVeedcRte2vLQmISQk5ZAsBQ2FFccuFdMK8Df93r9cIV50H2URkGVkXJU=
2022-12-03T14:37:53.395 INFO:teuthology.kill:  smithi182.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMV7iMVJjmvLnrb7uh5X7ZwMBpWQXOAuU7hzyk8N1zqRsWO0eOKaq3ZYJZ6zYeMdWdYD4Hog+qbXWXX7OrTue7oUxRNDwV+CRDqRuixjj9NSUTOVX2GPncYc8n3oIQDmSCL5KEBbStBaZB2qZ3NugNR62yGs1Ptw+bIrWpVztCM1NZ1OpfUsb1iTkxn4AG85B/3FZfvWYuM5kY2NDouWtf0kzBEkaPAQxcTt1igF6yxxH270TkjJM/lzsZE3gJe3g25syWjitCKxxc3UKOXsZ8czrMbR1rY6t+6lWO8O/w3cJ1ks4Fe5pa/94Se0EGa6yQMaoCKk7xAP5RFjw5/qUdpMcAlqeuUGdcpDIfIwWnjuHYoX+uD/+ctvVTtCpYZY7Q+LNcfNfH4Fa7Og/2kHU+O1/p1ynq1QAtFEAZW2JhMQ+JZqjp3EuASR32c/TMrV8v9L0kCI6oq/JBpGozr00PTsCOJyrZwypS5z52gKwMjV2Ygcvh4fT+SaAlKpdUZSM=
2022-12-03T14:37:53.395 INFO:teuthology.kill:2022-12-03 14:37:53,395.395 INFO:teuthology.nuke:Checking targets against current locks
2022-12-03T14:37:53.471 INFO:teuthology.kill:2022-12-03 14:37:53,470.470 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-12-03T14:37:53.502 INFO:teuthology.kill:2022-12-03 14:37:53,501.501 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-12-03T14:37:53.575 INFO:teuthology.kill:2022-12-03 14:37:53,572.572 INFO:teuthology.orchestra.console:Power off smithi174
2022-12-03T14:37:53.630 INFO:teuthology.kill:2022-12-03 14:37:53,630.630 INFO:teuthology.orchestra.console:Power off smithi182
2022-12-03T14:38:01.973 INFO:teuthology.kill:2022-12-03 14:38:01,972.972 INFO:teuthology.orchestra.console:Power off for smithi174 completed
2022-12-03T14:38:02.135 INFO:teuthology.kill:2022-12-03 14:38:02,134.134 INFO:teuthology.lock.ops:unlocked: smithi174.front.sepia.ceph.com
2022-12-03T14:38:06.139 INFO:teuthology.kill:2022-12-03 14:38:06,138.138 INFO:teuthology.orchestra.console:Power off for smithi182 completed
2022-12-03T14:38:06.303 INFO:teuthology.kill:2022-12-03 14:38:06,303.303 INFO:teuthology.lock.ops:unlocked: smithi182.front.sepia.ceph.com
2022-12-03T14:40:06.694 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-12-03T14:40:06.888 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi174 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-12-03_14:23:03-upgrade:nautilus-x-pacific-distro-default-smithi/7101994
2022-12-03T14:40:06.889 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi182 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-12-03_14:23:03-upgrade:nautilus-x-pacific-distro-default-smithi/7101994