2022-05-27T06:34:33.530 INFO:root:teuthology version: 1.1.1.dev341+g1b30281f 2022-05-27T06:34:33.774 INFO:teuthology.lock.ops:Start node 'smithi064.front.sepia.ceph.com' reimaging 2022-05-27T06:34:33.774 INFO:teuthology.lock.ops:Updating [smithi064.front.sepia.ceph.com]: reset os type and version on server 2022-05-27T06:34:33.775 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2022-05-27T06:34:33.818 INFO:teuthology.lock.ops:Node 'smithi064.front.sepia.ceph.com' reimaging is complete 2022-05-27T06:34:33.818 INFO:teuthology.lock.ops:Start node 'smithi071.front.sepia.ceph.com' reimaging 2022-05-27T06:34:33.818 INFO:teuthology.lock.ops:Updating [smithi071.front.sepia.ceph.com]: reset os type and version on server 2022-05-27T06:34:33.820 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2022-05-27T06:34:33.839 INFO:teuthology.lock.ops:Node 'smithi071.front.sepia.ceph.com' reimaging is complete 2022-05-27T06:34:33.839 INFO:teuthology.lock.ops:Start node 'smithi089.front.sepia.ceph.com' reimaging 2022-05-27T06:34:33.840 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: reset os type and version on server 2022-05-27T06:34:33.840 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2022-05-27T06:34:33.869 INFO:teuthology.lock.ops:Node 'smithi089.front.sepia.ceph.com' reimaging is complete 2022-05-27T06:34:34.039 INFO:teuthology.provision.fog.smithi064:Scheduling deploy of centos 8.stream 2022-05-27T06:34:34.067 INFO:teuthology.provision.fog.smithi071:Scheduling deploy of centos 8.stream 2022-05-27T06:34:34.120 INFO:teuthology.provision.fog.smithi089:Scheduling deploy of centos 8.stream 2022-05-27T06:34:34.351 INFO:teuthology.orchestra.console:Power off smithi064 2022-05-27T06:34:34.428 INFO:teuthology.orchestra.console:Power off smithi071 2022-05-27T06:34:34.512 INFO:teuthology.orchestra.console:Power off smithi089 2022-05-27T06:34:46.847 INFO:teuthology.orchestra.console:Power off for smithi064 completed 2022-05-27T06:34:46.915 INFO:teuthology.orchestra.console:Power off for smithi071 completed 2022-05-27T06:34:46.949 INFO:teuthology.orchestra.console:Power on smithi064 2022-05-27T06:34:46.989 INFO:teuthology.orchestra.console:Power off for smithi089 completed 2022-05-27T06:34:47.016 INFO:teuthology.orchestra.console:Power on smithi071 2022-05-27T06:34:47.091 INFO:teuthology.orchestra.console:Power on smithi089 2022-05-27T06:34:51.203 INFO:teuthology.orchestra.console:Power on for smithi064 completed 2022-05-27T06:34:51.304 INFO:teuthology.provision.fog.smithi064:Waiting for deploy to finish 2022-05-27T06:34:51.349 INFO:teuthology.orchestra.console:Power on for smithi089 completed 2022-05-27T06:34:51.450 INFO:teuthology.provision.fog.smithi089:Waiting for deploy to finish 2022-05-27T06:34:55.387 INFO:teuthology.orchestra.console:Power on for smithi071 completed 2022-05-27T06:34:55.488 INFO:teuthology.provision.fog.smithi071:Waiting for deploy to finish 2022-05-27T06:39:01.513 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-05-27T06:39:22.990 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-05-27T06:39:23.244 INFO:teuthology.provision.fog.smithi089:Node is ready 2022-05-27T06:39:23.303 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089.front.sepia.ceph.com 2022-05-27T06:39:23.362 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89 smithi089.front.sepia.ceph.com smithi089 2022-05-27T06:39:23.715 INFO:teuthology.provision.fog.smithi089:Deploy complete! 2022-05-27T06:39:27.207 INFO:teuthology.provision.fog.smithi071:Node is ready 2022-05-27T06:39:27.231 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071.front.sepia.ceph.com 2022-05-27T06:39:27.292 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71 smithi071.front.sepia.ceph.com smithi071 2022-05-27T06:39:27.670 INFO:teuthology.provision.fog.smithi071:Deploy complete! 2022-05-27T06:39:28.997 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-05-27T06:39:29.293 INFO:teuthology.provision.fog.smithi064:Node is ready 2022-05-27T06:39:29.315 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064.front.sepia.ceph.com 2022-05-27T06:39:29.376 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64 smithi064.front.sepia.ceph.com smithi064 2022-05-27T06:39:29.737 INFO:teuthology.provision.fog.smithi064:Deploy complete! 2022-05-27T06:39:30.098 INFO:teuthology.lock.ops:Checking smithi064.front.sepia.ceph.com 2022-05-27T06:39:30.099 INFO:teuthology.lock.ops:New key found. Updating... 2022-05-27T06:39:30.131 INFO:teuthology.lock.ops:Checking smithi071.front.sepia.ceph.com 2022-05-27T06:39:30.131 INFO:teuthology.lock.ops:New key found. Updating... 2022-05-27T06:39:30.160 INFO:teuthology.lock.ops:Checking smithi089.front.sepia.ceph.com 2022-05-27T06:39:30.161 INFO:teuthology.lock.ops:New key found. Updating... 2022-05-27T06:39:30.211 INFO:teuthology.lock.ops:Updating [smithi064.front.sepia.ceph.com]: set os type and version on server 2022-05-27T06:39:30.212 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2022-05-27T06:39:30.455 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2022-05-27T06:39:30.518 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2022-05-27T06:39:30.518 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="8" 2022-05-27T06:39:30.519 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2022-05-27T06:39:30.519 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2022-05-27T06:39:30.519 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="8" 2022-05-27T06:39:30.521 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el8" 2022-05-27T06:39:30.521 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 8" 2022-05-27T06:39:30.522 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2022-05-27T06:39:30.522 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-05-27T06:39:30.522 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2022-05-27T06:39:30.522 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-05-27T06:39:30.522 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-05-27T06:39:30.522 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-05-27T06:39:30.523 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2022-05-27T06:39:30.571 INFO:teuthology.lock.ops:Updating [smithi071.front.sepia.ceph.com]: set os type and version on server 2022-05-27T06:39:30.571 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2022-05-27T06:39:30.794 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2022-05-27T06:39:30.856 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2022-05-27T06:39:30.857 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="8" 2022-05-27T06:39:30.857 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2022-05-27T06:39:30.857 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2022-05-27T06:39:30.857 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="8" 2022-05-27T06:39:30.857 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el8" 2022-05-27T06:39:30.857 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 8" 2022-05-27T06:39:30.858 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2022-05-27T06:39:30.858 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-05-27T06:39:30.859 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2022-05-27T06:39:30.859 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-05-27T06:39:30.859 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-05-27T06:39:30.859 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-05-27T06:39:30.860 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2022-05-27T06:39:30.884 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: set os type and version on server 2022-05-27T06:39:30.885 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2022-05-27T06:39:31.101 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2022-05-27T06:39:31.159 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream" 2022-05-27T06:39:31.159 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8" 2022-05-27T06:39:31.159 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2022-05-27T06:39:31.159 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2022-05-27T06:39:31.160 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8" 2022-05-27T06:39:31.160 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8" 2022-05-27T06:39:31.160 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 8" 2022-05-27T06:39:31.160 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2022-05-27T06:39:31.160 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-05-27T06:39:31.160 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/" 2022-05-27T06:39:31.160 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-05-27T06:39:31.160 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-05-27T06:39:31.160 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-05-27T06:39:31.161 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2022-05-27T06:39:31.309 INFO:teuthology.dispatcher.supervisor:Running job 6850237 2022-05-27T06:39:31.310 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_1b30281f276b97a8186594b7f92fe1f728418ada/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2022-05-27_05:00:37-fs-wip-vshankar-testing-20220527-073645-testing-default-smithi/6850237 --name vshankar-2022-05-27_05:00:37-fs-wip-vshankar-testing-20220527-073645-testing-default-smithi --description fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} -- /home/teuthworker/archive/vshankar-2022-05-27_05:00:37-fs-wip-vshankar-testing-20220527-073645-testing-default-smithi/6850237/orig.config.yaml 2022-05-27T06:39:31.315 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2022-05-27_05:00:37-fs-wip-vshankar-testing-20220527-073645-testing-default-smithi/6850237 2022-05-27T06:39:31.323 INFO:teuthology.dispatcher.supervisor:Job PID: 26094 2022-05-27T06:39:31.323 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-05-27T13:10:03.607 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-05-27T13:10:03.992 INFO:teuthology.kill:Killing Pids: {26094} 2022-05-27T13:10:05.568 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'mds.b', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2022-05-27T13:10:05.568 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['client.0'] 2022-05-27T13:10:05.568 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['client.1'] 2022-05-27T13:10:05.568 INFO:teuthology.misc:Compressing logs... 2022-05-27T13:10:05.569 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2022-05-27T13:10:05.817 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2022-05-27T13:10:06.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2022-05-27T13:10:06.741 INFO:teuthology.misc:Compressing logs... 2022-05-27T13:10:07.155 INFO:teuthology.misc:Compressing logs... 2022-05-27T13:12:57.148 INFO:teuthology.kill:No teuthology processes running 2022-05-27T13:12:58.585 INFO:teuthology.kill:Nuking machines: ['smithi064', 'smithi071', 'smithi089'] 2022-05-27T13:12:59.819 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_1b30281f276b97a8186594b7f92fe1f728418ada/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-05-27T13:12:59.819 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-05-27T13:12:59.819 INFO:teuthology.kill:2022-05-27 13:12:59,818.818 INFO:teuthology.nuke:targets: 2022-05-27T13:12:59.819 INFO:teuthology.kill: smithi064.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCekW1Qn9nf6mdsEBIEjxWuT9DVEHBpy3gHzdZXdlEQXkleCUUwqDjJ3ke+93ROqU32CWhstOAE8kQucFQD+WmY17b0qWDPmexncxxn4VWm6DvxhxS3Shni/o1LJV7GvQg9UNOK9ZXozbvDPvFSAeCmSvA8EK10xsIuD7dSnV2o3eCIDAnPgs3OPjAJxqF7doMEbZVKRPgzDR7M86QHFxYc0+Mkf6cjkGUviwVtiPdf4KgyO3AMZxdpugBfl39er3Xlz5WyFPwiN1O4eUx4tnFZd/2iU8BWYUF3zLHjFXNRtxUIdtHCB08WtqCupWMFlmRp0tI0t3fGc/11cEgY1GU598VnYEU7QSpywWUrw6U1AkgvQWsgCF7LLF4Ko+qDftHbM3V/Yp8EfkTgZ3Js2lzlV+ZUZLlreb6AdYHpt/wvd4LxyyU3rBz6tY5142DLsfeUhxCVuC/r+GDhfYUMUZujbiiV+k+fulNpV4wgmkBdCoe+FWUQZsJXPsYKfVWbSX0= 2022-05-27T13:12:59.820 INFO:teuthology.kill: smithi071.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1+t7u+r81DpLOPvksTRJ2qN7IpLW+Yf5d7tTsYqq/q6IfG5qoEw+I+AkIETK6bO6BY7arfkEmcKYanGPjrFTCLz/ZZt9PrgZ0zi3sRr4tkP52WW09Iq7S4GiDiSihSugwkOenajzuKOEv/c53HFTMRtu2eN+wEnJj9UY8autwqo8an/ji2uija7ArqsP9Qrt/ixnns/OAmF0uynBMB86OOlmhokotdUJVN8CUZuuIDk3hTQ3iRl3I/9xS10BRvaK/xi2qoeO7cK9HK5ProvLRlp26Xl6HisASQuROYzo0tgPE2oyyh6jfNBij9YcIYZinsuX0xl0Uk9eiZMgHHsxW417vo3zggGdLAhvSITnNkAtVBqIDfKVIfROVSjhNEpYIvvqPnaC2IH/b9pgI6fK5xqIl58nlB2aZRVP9DPdGC2FHR4LA5EIIy9w5Gt7f3igEtEILYPzZz3jd+prcBiOHFlCOU7oUZhbeqIigbT33WRu5+vRXeV3RKTWq99mxpB8= 2022-05-27T13:12:59.820 INFO:teuthology.kill: smithi089.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ32UdORnb6qulQrimPbf2svf8NWK5ZIg2QFC5l2N7E//XLiUATWjFfpxQW3Jo8I793UfCASe5QCbUF777H28KaV9Z94ZdQgjYDr3AEEVXJMf+567+6mKjUVcnSJVAJThs8wbSyB+x77D9wr5GVKatx4a12/0ZTolha+C4SpsW0Cr0hIovZgaYgeWbSa/FHiNLRj3aZdLgnmFucAiXRnDrIlnbg59JKuOsvCRzHvZ3lC8hH7vNox+sOF+9ycORMmxtsEw2+BUiL2cNLFQ9ztlokUrZujcsK0ztDpi/dfj63r/0Y/77ocIfEbyECKnjB+cZSgrr3YfHPPB96YMy49jTd4svUjAqjCI9AeQg1VuvvIrYZOrtKe1aw+OOA84pDey+BG8+CirnBzRGdEIHqmCTocftYKkhWqwxbN5ER03mXottnruwTEWVWs25AOXnMrOJXU4Mh0Y3wrdXNumnJ/0n1ZKd25mVJWRY2Ly4amEaCdIy2gs4itIocrfuXfZq/BE= 2022-05-27T13:12:59.855 INFO:teuthology.kill:2022-05-27 13:12:59,854.854 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-05-27T13:12:59.856 INFO:teuthology.kill:2022-05-27 13:12:59,856.856 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-05-27T13:12:59.860 INFO:teuthology.kill:2022-05-27 13:12:59,860.860 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-05-27T13:12:59.948 INFO:teuthology.kill:2022-05-27 13:12:59,948.948 INFO:teuthology.orchestra.console:Power off smithi064 2022-05-27T13:12:59.966 INFO:teuthology.kill:2022-05-27 13:12:59,966.966 INFO:teuthology.orchestra.console:Power off smithi071 2022-05-27T13:12:59.979 INFO:teuthology.kill:2022-05-27 13:12:59,978.978 INFO:teuthology.orchestra.console:Power off smithi089 2022-05-27T13:13:08.349 INFO:teuthology.kill:2022-05-27 13:13:08,349.349 INFO:teuthology.orchestra.console:Power off for smithi064 completed 2022-05-27T13:13:08.357 INFO:teuthology.kill:2022-05-27 13:13:08,356.356 INFO:teuthology.orchestra.console:Power off for smithi089 completed 2022-05-27T13:13:08.516 INFO:teuthology.kill:2022-05-27 13:13:08,515.515 INFO:teuthology.lock.ops:unlocked: smithi064.front.sepia.ceph.com 2022-05-27T13:13:08.562 INFO:teuthology.kill:2022-05-27 13:13:08,518.518 INFO:teuthology.lock.ops:unlocked: smithi089.front.sepia.ceph.com 2022-05-27T13:13:12.456 INFO:teuthology.kill:2022-05-27 13:13:12,455.455 INFO:teuthology.orchestra.console:Power off for smithi071 completed 2022-05-27T13:13:12.610 INFO:teuthology.kill:2022-05-27 13:13:12,610.610 INFO:teuthology.lock.ops:unlocked: smithi071.front.sepia.ceph.com 2022-05-27T13:15:12.991 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-05-27T13:15:13.142 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi064 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-05-27_03:20:03-krbd-quincy-testing-default-smithi/6849906 2022-05-27T13:15:13.142 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi071 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-05-27_03:20:03-krbd-quincy-testing-default-smithi/6849906 2022-05-27T13:15:13.142 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi089 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-05-27_03:20:03-krbd-quincy-testing-default-smithi/6849906