2022-06-02T13:36:34.525 INFO:root:teuthology version: 1.1.1.dev360+g53fc7fef 2022-06-02T13:36:34.698 INFO:teuthology.lock.ops:Start node 'smithi046.front.sepia.ceph.com' reimaging 2022-06-02T13:36:34.699 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: reset os type and version on server 2022-06-02T13:36:34.699 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2022-06-02T13:36:34.722 INFO:teuthology.lock.ops:Node 'smithi046.front.sepia.ceph.com' reimaging is complete 2022-06-02T13:36:34.723 INFO:teuthology.lock.ops:Start node 'smithi135.front.sepia.ceph.com' reimaging 2022-06-02T13:36:34.723 INFO:teuthology.lock.ops:Updating [smithi135.front.sepia.ceph.com]: reset os type and version on server 2022-06-02T13:36:34.723 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2022-06-02T13:36:34.748 INFO:teuthology.lock.ops:Node 'smithi135.front.sepia.ceph.com' reimaging is complete 2022-06-02T13:36:34.748 INFO:teuthology.lock.ops:Start node 'smithi190.front.sepia.ceph.com' reimaging 2022-06-02T13:36:34.748 INFO:teuthology.lock.ops:Updating [smithi190.front.sepia.ceph.com]: reset os type and version on server 2022-06-02T13:36:34.748 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2022-06-02T13:36:34.768 INFO:teuthology.lock.ops:Node 'smithi190.front.sepia.ceph.com' reimaging is complete 2022-06-02T13:36:34.973 INFO:teuthology.provision.fog.smithi135:Scheduling deploy of centos 8.stream 2022-06-02T13:36:35.005 INFO:teuthology.provision.fog.smithi046:Scheduling deploy of centos 8.stream 2022-06-02T13:36:35.011 INFO:teuthology.provision.fog.smithi190:Scheduling deploy of centos 8.stream 2022-06-02T13:36:35.545 INFO:teuthology.orchestra.console:Power off smithi046 2022-06-02T13:36:35.564 INFO:teuthology.orchestra.console:Power off smithi135 2022-06-02T13:36:35.656 INFO:teuthology.orchestra.console:Power off smithi190 2022-06-02T13:36:35.699 INFO:teuthology.orchestra.console:Power off for smithi046 completed 2022-06-02T13:36:35.717 INFO:teuthology.orchestra.console:Power off for smithi135 completed 2022-06-02T13:36:35.801 INFO:teuthology.orchestra.console:Power on smithi046 2022-06-02T13:36:35.813 INFO:teuthology.orchestra.console:Power off for smithi190 completed 2022-06-02T13:36:35.818 INFO:teuthology.orchestra.console:Power on smithi135 2022-06-02T13:36:35.917 INFO:teuthology.orchestra.console:Power on smithi190 2022-06-02T13:36:40.174 INFO:teuthology.orchestra.console:Power on for smithi190 completed 2022-06-02T13:36:40.275 INFO:teuthology.provision.fog.smithi190:Waiting for deploy to finish 2022-06-02T13:36:44.187 INFO:teuthology.orchestra.console:Power on for smithi046 completed 2022-06-02T13:36:44.264 INFO:teuthology.orchestra.console:Power on for smithi135 completed 2022-06-02T13:36:44.288 INFO:teuthology.provision.fog.smithi046:Waiting for deploy to finish 2022-06-02T13:36:44.366 INFO:teuthology.provision.fog.smithi135:Waiting for deploy to finish 2022-06-02T13:40:44.561 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-06-02T13:41:04.406 INFO:teuthology.provision.fog.smithi046:Node is ready 2022-06-02T13:41:04.453 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046.front.sepia.ceph.com 2022-06-02T13:41:04.526 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2022-06-02T13:41:04.963 INFO:teuthology.provision.fog.smithi046:Deploy complete! 2022-06-02T13:41:13.073 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-06-02T13:41:13.362 INFO:teuthology.provision.fog.smithi190:Node is ready 2022-06-02T13:41:13.440 INFO:teuthology.orchestra.run.smithi190.stdout:smithi190.front.sepia.ceph.com 2022-06-02T13:41:13.506 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.15.190 smithi190.front.sepia.ceph.com smithi190 2022-06-02T13:41:13.923 INFO:teuthology.provision.fog.smithi190:Deploy complete! 2022-06-02T13:41:17.853 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-06-02T13:41:18.078 INFO:teuthology.provision.fog.smithi135:Node is ready 2022-06-02T13:41:18.163 INFO:teuthology.orchestra.run.smithi135.stdout:smithi135.front.sepia.ceph.com 2022-06-02T13:41:18.367 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135 smithi135.front.sepia.ceph.com smithi135 2022-06-02T13:41:18.995 INFO:teuthology.provision.fog.smithi135:Deploy complete! 2022-06-02T13:41:19.945 INFO:teuthology.lock.ops:Checking smithi046.front.sepia.ceph.com 2022-06-02T13:41:19.998 INFO:teuthology.lock.ops:New key found. Updating... 2022-06-02T13:41:20.144 INFO:teuthology.lock.ops:Checking smithi135.front.sepia.ceph.com 2022-06-02T13:41:20.144 INFO:teuthology.lock.ops:New key found. Updating... 2022-06-02T13:41:20.195 INFO:teuthology.lock.ops:Checking smithi190.front.sepia.ceph.com 2022-06-02T13:41:20.195 INFO:teuthology.lock.ops:New key found. Updating... 2022-06-02T13:41:20.306 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: set os type and version on server 2022-06-02T13:41:20.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2022-06-02T13:41:21.554 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2022-06-02T13:41:21.618 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2022-06-02T13:41:21.618 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="8" 2022-06-02T13:41:21.618 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2022-06-02T13:41:21.618 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2022-06-02T13:41:21.618 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="8" 2022-06-02T13:41:21.618 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el8" 2022-06-02T13:41:21.618 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 8" 2022-06-02T13:41:21.619 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2022-06-02T13:41:21.619 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-06-02T13:41:21.619 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2022-06-02T13:41:21.619 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-06-02T13:41:21.645 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-06-02T13:41:21.646 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-06-02T13:41:21.647 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2022-06-02T13:41:21.872 INFO:teuthology.lock.ops:Updating [smithi135.front.sepia.ceph.com]: set os type and version on server 2022-06-02T13:41:21.979 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2022-06-02T13:41:23.000 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2022-06-02T13:41:23.158 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="CentOS Stream" 2022-06-02T13:41:23.159 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="8" 2022-06-02T13:41:23.159 INFO:teuthology.orchestra.run.smithi135.stdout:ID="centos" 2022-06-02T13:41:23.159 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="rhel fedora" 2022-06-02T13:41:23.179 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="8" 2022-06-02T13:41:23.179 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el8" 2022-06-02T13:41:23.179 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="CentOS Stream 8" 2022-06-02T13:41:23.179 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2022-06-02T13:41:23.179 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-06-02T13:41:23.190 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://centos.org/" 2022-06-02T13:41:23.190 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-06-02T13:41:23.190 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-06-02T13:41:23.190 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-06-02T13:41:23.196 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2022-06-02T13:41:23.349 INFO:teuthology.lock.ops:Updating [smithi190.front.sepia.ceph.com]: set os type and version on server 2022-06-02T13:41:23.420 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2022-06-02T13:41:24.651 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2022-06-02T13:41:24.736 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2022-06-02T13:41:24.737 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="8" 2022-06-02T13:41:24.737 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2022-06-02T13:41:24.737 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2022-06-02T13:41:24.737 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="8" 2022-06-02T13:41:24.737 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el8" 2022-06-02T13:41:24.738 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 8" 2022-06-02T13:41:24.738 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2022-06-02T13:41:24.738 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-06-02T13:41:24.738 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2022-06-02T13:41:24.739 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-06-02T13:41:24.739 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-06-02T13:41:24.739 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-06-02T13:41:24.740 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2022-06-02T13:41:25.461 INFO:teuthology.dispatcher.supervisor:Running job 6860313 2022-06-02T13:41:25.461 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_53fc7fef6ccaf21fabe4a23a38668920f3ae41b8/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2022-06-02_13:35:44-fs-wip-vshankar-testing-20220527-073645-distro-basic-smithi/6860313 --name rishabh-2022-06-02_13:35:44-fs-wip-vshankar-testing-20220527-073645-distro-basic-smithi --description fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} -- /home/teuthworker/archive/rishabh-2022-06-02_13:35:44-fs-wip-vshankar-testing-20220527-073645-distro-basic-smithi/6860313/orig.config.yaml 2022-06-02T13:41:25.544 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2022-06-02_13:35:44-fs-wip-vshankar-testing-20220527-073645-distro-basic-smithi/6860313 2022-06-02T13:41:25.591 INFO:teuthology.dispatcher.supervisor:Job PID: 25069 2022-06-02T13:41:25.591 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-06-02T20:11:56.846 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-06-02T20:11:57.076 INFO:teuthology.kill:Killing Pids: {25069} 2022-06-02T20:11:58.514 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'mds.b', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2022-06-02T20:11:58.514 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['client.0'] 2022-06-02T20:11:58.514 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['client.1'] 2022-06-02T20:11:58.515 INFO:teuthology.misc:Compressing logs... 2022-06-02T20:11:58.515 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2022-06-02T20:11:58.739 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2022-06-02T20:11:58.956 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2022-06-02T20:11:59.345 INFO:teuthology.misc:Compressing logs... 2022-06-02T20:11:59.512 INFO:teuthology.misc:Compressing logs... 2022-06-02T20:14:34.191 INFO:teuthology.kill:No teuthology processes running 2022-06-02T20:14:35.521 INFO:teuthology.kill:Nuking machines: ['smithi046', 'smithi135', 'smithi190'] 2022-06-02T20:14:36.689 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_53fc7fef6ccaf21fabe4a23a38668920f3ae41b8/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-06-02T20:14:36.689 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-06-02T20:14:36.690 INFO:teuthology.kill:2022-06-02 20:14:36,688.688 INFO:teuthology.nuke:targets: 2022-06-02T20:14:36.690 INFO:teuthology.kill: smithi046.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7fT4CYhhAjBTLzw5V7oCJgTwSJfe53mcLrFQ8Z68kiOROd2ItqErfKBE6nTkIiKBWEwwnNJoqURsT+o5RWdL4k7amNvggeSnT/wySMXaPZAfwtWfst580XqB4SkeGIKwHCHzvo4ev7HJctF0nzK3M7NoQOcish6+bQOrxXkG0ulnbltmIo8EC8ap1XjlR263Oc1rBjjAhr0sMeoOAmmWJ48W7+0fdk/K1MyMZXrNAnnDdIC2bS++VxAZvbcSdh+0jVGPbyyTr1LIOghI8BvcMcm0vknMTUcFwXlDWWpx/s4o7UR8Lm4PipnVsFuacarKhanhVK38z9U/ydGj2ISN9mbyVqA9WdnKVvlz/nc5JZxLCcPKdOpMzSCTY8gMjeuVIPFe7H6VwuguLw7Wf+GNpX2pFTNMfSNGDJH5UUhi5X2awr0ZJmN+loL5ozFBxDXbfnRHWF/sO1EltcdWPocDDZCQQqXUzKPXX8MIbQYPSsSgtVMQcW+WmyvwqHP+l0WU= 2022-06-02T20:14:36.690 INFO:teuthology.kill: smithi135.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3b2OD771IoWyHWZCQDf75aLRvOZoym0upylcuAVbLHwcUGiCE03Vt7JuGNdsRP/tLYdZK9Pr/S1iXAByyBPRSXwdQqIZOcgvfBlbFUigAj84dHxWew1nWPaE/Tb9D3xgOFwCm+7hm9L5IwVF+aIdRYwEe5o0AtCZw1a3KvCsPbKjdxiWWXy/yIu9TWN2MuVhnWF2WOTkd8GEocZ8gqatATEeSaYhi/WUKtu3Mfk/9JQLyqO3q2J0Zy8BnamDSUAOpvHjxZaajvDSM7jiQassLxnHWg8ViSxeEcC6lMITW7BPKyRdfXxeJsfYRPJsIsCuu3xK50yxpcJPvZNgfHxjXXYFel2TdhqXyHxRwkJS1co0xTTIOU7X0DnKKLSvFJAlnlM9resjxpQoEMBfQ084Amli1KF/R7WxvzhUhOYUlAnKekrkjK7DhRyE1jko5NzIaqAC01vmhZEbptrs563OSsbTlxKFx9VvPH+x+LZ8jGfV7toPJB9WmFGJDH0pwk0c= 2022-06-02T20:14:36.690 INFO:teuthology.kill: smithi190.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC20s94WzPJfoPcLFUpmLBIJ1vXN4e+//DY3vsteG6q+mgkdYLWQkaGQgzG+tklNQc6i78VeeSANRAKNXD+TGCjl2EDXbJ10ZNbtcDLosKvHb/evEOJM9EZM7y82kpxs4Bik1w5X/4mErOh82Z3nYUIl5F/T5F2RVS1U1km+P9Goa1Vs3NgWoABKl47HeiFo4hEfrNLd2vo9Yb0CPCrnsClsfIO/go23KVpa6UkptIrmnb2LlxASZ8iZVFz5qeSk/R0s85Bacc5r4uc15th0s73EYUs644GxWMYQ4HvKJCmC0LiYRpqhGgBBi5iRlDHuY8k9PiOIX9zqSk5DYrj2LU3W5AN54UiLePr2g3w/DrXZ1AHyVYiZ9QZKAmrCBeQoXLP1KWrv7pjOTvhKm3m+5RQIQUG+wJLCunUXfXpmvj+OvLtOdhqak2RCa4XWCnTM+1SXC4jo8FW8kZcdzs9v2PWCw07s/E3D1NLMdI+15ia72NJs4+Oo/NyjVLRkPQl0Ws= 2022-06-02T20:14:36.735 INFO:teuthology.kill:2022-06-02 20:14:36,735.735 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-02T20:14:36.745 INFO:teuthology.kill:2022-06-02 20:14:36,745.745 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-02T20:14:36.806 INFO:teuthology.kill:2022-06-02 20:14:36,805.805 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-06-02T20:14:36.850 INFO:teuthology.kill:2022-06-02 20:14:36,849.849 INFO:teuthology.orchestra.console:Power off smithi190 2022-06-02T20:14:36.867 INFO:teuthology.kill:2022-06-02 20:14:36,867.867 INFO:teuthology.orchestra.console:Power off smithi135 2022-06-02T20:14:36.905 INFO:teuthology.kill:2022-06-02 20:14:36,905.905 INFO:teuthology.orchestra.console:Power off smithi046 2022-06-02T20:14:45.237 INFO:teuthology.kill:2022-06-02 20:14:45,236.236 INFO:teuthology.orchestra.console:Power off for smithi190 completed 2022-06-02T20:14:45.275 INFO:teuthology.kill:2022-06-02 20:14:45,274.274 INFO:teuthology.orchestra.console:Power off for smithi046 completed 2022-06-02T20:14:45.397 INFO:teuthology.kill:2022-06-02 20:14:45,395.395 INFO:teuthology.lock.ops:unlocked: smithi190.front.sepia.ceph.com 2022-06-02T20:14:45.436 INFO:teuthology.kill:2022-06-02 20:14:45,436.436 INFO:teuthology.lock.ops:unlocked: smithi046.front.sepia.ceph.com 2022-06-02T20:14:49.355 INFO:teuthology.kill:2022-06-02 20:14:49,355.355 INFO:teuthology.orchestra.console:Power off for smithi135 completed 2022-06-02T20:14:49.503 INFO:teuthology.kill:2022-06-02 20:14:49,503.503 INFO:teuthology.lock.ops:unlocked: smithi135.front.sepia.ceph.com 2022-06-02T20:16:49.813 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-06-02T20:16:49.964 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi046 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-05-31_05:15:02-krbd-octopus-testing-default-smithi/6855324 2022-06-02T20:16:49.964 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi135 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-05-31_05:15:02-krbd-octopus-testing-default-smithi/6855325 2022-06-02T20:16:49.965 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi190 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-05-31_05:15:02-krbd-octopus-testing-default-smithi/6855324