2022-05-08T08:16:06.750 INFO:root:teuthology version: 1.1.1.dev333+gb018fa01 2022-05-08T08:16:06.899 INFO:teuthology.lock.ops:Start node 'smithi160.front.sepia.ceph.com' reimaging 2022-05-08T08:16:06.901 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: reset os type and version on server 2022-05-08T08:16:06.901 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2022-05-08T08:16:06.925 INFO:teuthology.lock.ops:Node 'smithi160.front.sepia.ceph.com' reimaging is complete 2022-05-08T08:16:06.926 INFO:teuthology.lock.ops:Start node 'smithi169.front.sepia.ceph.com' reimaging 2022-05-08T08:16:06.926 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: reset os type and version on server 2022-05-08T08:16:06.927 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2022-05-08T08:16:06.949 INFO:teuthology.lock.ops:Node 'smithi169.front.sepia.ceph.com' reimaging is complete 2022-05-08T08:16:07.143 INFO:teuthology.provision.fog.smithi160:Scheduling deploy of centos 8.stream 2022-05-08T08:16:07.166 INFO:teuthology.provision.fog.smithi169:Scheduling deploy of centos 8.stream 2022-05-08T08:16:08.152 INFO:teuthology.orchestra.console:Power off smithi160 2022-05-08T08:16:08.247 INFO:teuthology.orchestra.console:Power off smithi169 2022-05-08T08:16:08.293 INFO:teuthology.orchestra.console:Power off for smithi160 completed 2022-05-08T08:16:08.388 INFO:teuthology.orchestra.console:Power off for smithi169 completed 2022-05-08T08:16:08.395 INFO:teuthology.orchestra.console:Power on smithi160 2022-05-08T08:16:08.489 INFO:teuthology.orchestra.console:Power on smithi169 2022-05-08T08:16:12.648 INFO:teuthology.orchestra.console:Power on for smithi160 completed 2022-05-08T08:16:12.738 INFO:teuthology.orchestra.console:Power on for smithi169 completed 2022-05-08T08:16:12.750 INFO:teuthology.provision.fog.smithi160:Waiting for deploy to finish 2022-05-08T08:16:12.840 INFO:teuthology.provision.fog.smithi169:Waiting for deploy to finish 2022-05-08T08:21:07.021 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-05-08T08:21:07.313 INFO:teuthology.provision.fog.smithi169:Node is ready 2022-05-08T08:21:07.331 INFO:teuthology.orchestra.run.smithi169.stdout:smithi169.front.sepia.ceph.com 2022-05-08T08:21:07.391 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.15.169 smithi169.front.sepia.ceph.com smithi169 2022-05-08T08:21:07.755 INFO:teuthology.provision.fog.smithi169:Deploy complete! 2022-05-08T08:21:09.351 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-05-08T08:21:09.626 INFO:teuthology.provision.fog.smithi160:Node is ready 2022-05-08T08:21:09.685 INFO:teuthology.orchestra.run.smithi160.stdout:smithi160.front.sepia.ceph.com 2022-05-08T08:21:09.743 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160 smithi160.front.sepia.ceph.com smithi160 2022-05-08T08:21:10.113 INFO:teuthology.provision.fog.smithi160:Deploy complete! 2022-05-08T08:21:10.265 INFO:teuthology.lock.ops:Checking smithi160.front.sepia.ceph.com 2022-05-08T08:21:10.266 INFO:teuthology.lock.ops:New key found. Updating... 2022-05-08T08:21:10.284 INFO:teuthology.lock.ops:Checking smithi169.front.sepia.ceph.com 2022-05-08T08:21:10.285 INFO:teuthology.lock.ops:New key found. Updating... 2022-05-08T08:21:10.304 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: set os type and version on server 2022-05-08T08:21:10.304 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2022-05-08T08:21:10.504 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2022-05-08T08:21:10.564 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream" 2022-05-08T08:21:10.565 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="8" 2022-05-08T08:21:10.565 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2022-05-08T08:21:10.565 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2022-05-08T08:21:10.565 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="8" 2022-05-08T08:21:10.566 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el8" 2022-05-08T08:21:10.566 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 8" 2022-05-08T08:21:10.566 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2022-05-08T08:21:10.566 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-05-08T08:21:10.567 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/" 2022-05-08T08:21:10.567 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-05-08T08:21:10.567 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-05-08T08:21:10.567 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-05-08T08:21:10.569 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2022-05-08T08:21:10.587 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: set os type and version on server 2022-05-08T08:21:10.587 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2022-05-08T08:21:10.850 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2022-05-08T08:21:10.911 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2022-05-08T08:21:10.912 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="8" 2022-05-08T08:21:10.912 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2022-05-08T08:21:10.912 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2022-05-08T08:21:10.913 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="8" 2022-05-08T08:21:10.913 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el8" 2022-05-08T08:21:10.913 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 8" 2022-05-08T08:21:10.913 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2022-05-08T08:21:10.914 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-05-08T08:21:10.914 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2022-05-08T08:21:10.914 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-05-08T08:21:10.914 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-05-08T08:21:10.915 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-05-08T08:21:10.916 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2022-05-08T08:21:11.024 INFO:teuthology.dispatcher.supervisor:Running job 6827205 2022-05-08T08:21:11.025 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b018fa018d3f885cbc018f839b42536477653f84/virtualenv/bin/teuthology -v --owner scheduled_ozeneva@teuthology --archive /home/teuthworker/archive/ozeneva-2022-05-08_08:15:09-orch:cephadm-wip-omri-cephadm-tracing-distro-basic-smithi/6827205 --name ozeneva-2022-05-08_08:15:09-orch:cephadm-wip-omri-cephadm-tracing-distro-basic-smithi --description orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/ozeneva-2022-05-08_08:15:09-orch:cephadm-wip-omri-cephadm-tracing-distro-basic-smithi/6827205/orig.config.yaml 2022-05-08T08:21:11.028 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ozeneva-2022-05-08_08:15:09-orch:cephadm-wip-omri-cephadm-tracing-distro-basic-smithi/6827205 2022-05-08T08:21:11.030 INFO:teuthology.dispatcher.supervisor:Job PID: 27254 2022-05-08T08:21:11.031 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-05-08T14:51:39.731 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-05-08T14:51:39.895 INFO:teuthology.kill:Killing Pids: {27254} 2022-05-08T14:51:41.414 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-05-08T14:51:41.415 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-05-08T14:51:41.415 INFO:teuthology.misc:Compressing logs... 2022-05-08T14:51:41.416 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2022-05-08T14:51:41.622 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2022-05-08T14:51:41.690 INFO:teuthology.orchestra.run.smithi160.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-05-08T14:51:41.848 INFO:teuthology.orchestra.run.smithi169.stderr:find: ‘/var/lib/ceph/crash’: No such file or directory 2022-05-08T14:51:41.963 INFO:teuthology.orchestra.run.smithi160.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-05-08T14:51:41.964 INFO:teuthology.orchestra.run.smithi160.stderr:tar: Error is not recoverable: exiting now 2022-05-08T14:51:42.002 INFO:teuthology.orchestra.run.smithi169.stderr:tar: /var/lib/ceph/crash: Cannot open: No such file or directory 2022-05-08T14:51:42.002 INFO:teuthology.orchestra.run.smithi169.stderr:tar: Error is not recoverable: exiting now 2022-05-08T14:51:42.003 INFO:teuthology.misc:Compressing logs... 2022-05-08T14:51:42.307 INFO:teuthology.misc:Compressing logs... 2022-05-08T14:51:45.398 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/6b1574e6-cea8-11ec-8c39-001a4aab830c/ceph-mgr.smithi160.kquwpi.log: file size changed while zipping 2022-05-08T14:51:47.092 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/6b1574e6-cea8-11ec-8c39-001a4aab830c/ceph-mon.smithi169.log: file size changed while zipping 2022-05-08T14:52:12.416 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/6b1574e6-cea8-11ec-8c39-001a4aab830c/ceph-osd.3.log: file size changed while zipping 2022-05-08T14:52:13.480 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/6b1574e6-cea8-11ec-8c39-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2022-05-08T14:52:35.804 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/6b1574e6-cea8-11ec-8c39-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2022-05-08T14:52:38.579 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/6b1574e6-cea8-11ec-8c39-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2022-05-08T14:52:54.762 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/6b1574e6-cea8-11ec-8c39-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2022-05-08T14:52:58.422 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/6b1574e6-cea8-11ec-8c39-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2022-05-08T14:53:07.978 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/6b1574e6-cea8-11ec-8c39-001a4aab830c/ceph-mon.smithi160.log: file size changed while zipping 2022-05-08T14:53:30.585 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/6b1574e6-cea8-11ec-8c39-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2022-05-08T14:53:30.848 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/6b1574e6-cea8-11ec-8c39-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2022-05-08T14:54:13.431 INFO:teuthology.kill:No teuthology processes running 2022-05-08T14:54:14.759 INFO:teuthology.kill:Nuking machines: ['smithi160', 'smithi169'] 2022-05-08T14:54:15.901 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b018fa018d3f885cbc018f839b42536477653f84/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-05-08T14:54:15.902 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-05-08T14:54:15.902 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_b018fa018d3f885cbc018f839b42536477653f84/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated 2022-05-08T14:54:15.903 INFO:teuthology.kill: "class": algorithms.Blowfish, 2022-05-08T14:54:15.903 INFO:teuthology.kill:2022-05-08 14:54:15,901.901 INFO:teuthology.nuke:targets: 2022-05-08T14:54:15.903 INFO:teuthology.kill: smithi160.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6vCHBKLEK8bqUOBsqYIMFBT5q1s3VThJFUlYtJ9a70t10e34y+j9FwFDUzSnjKF1ZUahUQmxgCWJRULhpLylpUXmWYQ/SkBDNiAJdKKjZX10CBSMdC9vDUKQzD0g7l1Y5c3mJah2rP2DJFP5y6gPKjCFzCClpgtbaqDcvpEoLW4/QurDGVjnEBez7hiUm7Qptd6qeoANef5Zs2XSGHTPR8m3g/WmwBvXGzuxkB9s9pM0Ve5BwwcfJRwsjwZ1PF6OrELgx75I3g13z36fjjzG75kvt8NvOullfRge+JjjirA0Fq0FbKa1SYM4aXiVyc5ml5MpqF2RwK0z34w8ohvedcxAbAflLetoy0MGh7PuT8D4rpisiRn3RMg4ky0cmU+ZGJmfmlJjjqAAyJf31+e2WLsl+d9Y4ZmTrLB7IduSPV1PgeLi1JinTgLvFk3OSyHw8HVs5ddOQAkAZ0mLLq7TZWdSuBxcQYcTltkY8xwKNfpLXM2ApLil1PrRd7S/lOOM= 2022-05-08T14:54:15.903 INFO:teuthology.kill: smithi169.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDsfAiUtdgRwbOngToNfS4/PC/XZq5FtIu9lg64lRP0BdoDkr7TWmTls4coLjzJnmhxJBG7pHAk2oiQZ0THTpW/Ag7aAErZ7hnS+k2MEXWAWsInUv8/GlgdEwA0gzKMv1q3WHm2oJbBi6r2fnaP3/o2G7W4gIhCjO5E2626A2d4Cy/k3eYk5u3+AuJEUDvGEzTIH0jMhq0ZmZk6nj449l0w7oNsWzOSdJAEkIgNz0eIczAzvMZw5UeG/0SRIpoPWi0hkDYkGTUj87HyCNG3eC+T+uniXckTd2e40scNWlhgPsv5BEK9qmTqQW3EjCQMk3xEy3zd+IXrv3kkyAuY59G7DzoJNRNaZGeaEsZbE2YscksbbfWEJlTN7Kyd+mgn5Oa3JNzHQhz25F/lFXSknt2kzttgg9pc62TjG6xJZFYVeqVNBqpX95uv3E85lquMaXKlnmjXhZe3hgkBem5pIL13iPewC6va0sr6FQMPVsad4S9G/l4YzUHowKhQXgE2sX0= 2022-05-08T14:54:15.929 INFO:teuthology.kill:2022-05-08 14:54:15,928.928 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-05-08T14:54:15.931 INFO:teuthology.kill:2022-05-08 14:54:15,931.931 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-05-08T14:54:16.000 INFO:teuthology.kill:2022-05-08 14:54:16,000.000 INFO:teuthology.orchestra.console:Power off smithi169 2022-05-08T14:54:16.015 INFO:teuthology.kill:2022-05-08 14:54:16,015.015 INFO:teuthology.orchestra.console:Power off smithi160 2022-05-08T14:54:28.481 INFO:teuthology.kill:2022-05-08 14:54:28,481.481 INFO:teuthology.orchestra.console:Power off for smithi169 completed 2022-05-08T14:54:28.486 INFO:teuthology.kill:2022-05-08 14:54:28,485.485 INFO:teuthology.orchestra.console:Power off for smithi160 completed 2022-05-08T14:54:28.629 INFO:teuthology.kill:2022-05-08 14:54:28,628.628 INFO:teuthology.lock.ops:unlocked: smithi169.front.sepia.ceph.com 2022-05-08T14:54:28.633 INFO:teuthology.kill:2022-05-08 14:54:28,633.633 INFO:teuthology.lock.ops:unlocked: smithi160.front.sepia.ceph.com 2022-05-08T14:56:28.941 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-05-08T14:56:29.042 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi160 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-05-03_03:15:03-fs-quincy-distro-default-smithi/6818943 2022-05-08T14:56:29.042 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi169 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-05-03_03:15:03-fs-quincy-distro-default-smithi/6818943