2022-12-05T10:07:24.339 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-12-05T10:07:24.643 INFO:teuthology.lock.ops:Start node 'smithi136.front.sepia.ceph.com' reimaging 2022-12-05T10:07:24.646 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: reset os type and version on server 2022-12-05T10:07:24.646 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2022-12-05T10:07:24.686 INFO:teuthology.lock.ops:Node 'smithi136.front.sepia.ceph.com' reimaging is complete 2022-12-05T10:07:24.687 INFO:teuthology.lock.ops:Start node 'smithi143.front.sepia.ceph.com' reimaging 2022-12-05T10:07:24.687 INFO:teuthology.lock.ops:Updating [smithi143.front.sepia.ceph.com]: reset os type and version on server 2022-12-05T10:07:24.688 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2022-12-05T10:07:24.723 INFO:teuthology.lock.ops:Node 'smithi143.front.sepia.ceph.com' reimaging is complete 2022-12-05T10:07:24.937 INFO:teuthology.provision.fog.smithi136:Scheduling deploy of centos 8.stream 2022-12-05T10:07:24.949 INFO:teuthology.provision.fog.smithi143:Scheduling deploy of centos 8.stream 2022-12-05T10:07:25.352 INFO:teuthology.orchestra.console:Power off smithi143 2022-12-05T10:07:25.425 INFO:teuthology.orchestra.console:Power off smithi136 2022-12-05T10:07:33.901 INFO:teuthology.orchestra.console:Power off for smithi143 completed 2022-12-05T10:07:34.002 INFO:teuthology.orchestra.console:Power on smithi143 2022-12-05T10:07:38.087 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2022-12-05T10:07:38.199 INFO:teuthology.orchestra.console:Power on smithi136 2022-12-05T10:07:42.494 INFO:teuthology.orchestra.console:Power on for smithi143 completed 2022-12-05T10:07:42.548 INFO:teuthology.orchestra.console:Power on for smithi136 completed 2022-12-05T10:07:42.596 INFO:teuthology.provision.fog.smithi143:Waiting for deploy to finish 2022-12-05T10:07:42.650 INFO:teuthology.provision.fog.smithi136:Waiting for deploy to finish 2022-12-05T10:12:20.787 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-12-05T10:12:20.816 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-12-05T10:12:21.052 INFO:teuthology.provision.fog.smithi136:Node is ready 2022-12-05T10:12:21.066 INFO:teuthology.provision.fog.smithi143:Node is ready 2022-12-05T10:12:21.072 INFO:teuthology.orchestra.run.smithi136.stdout:smithi136.front.sepia.ceph.com 2022-12-05T10:12:21.084 INFO:teuthology.orchestra.run.smithi143.stdout:smithi143.front.sepia.ceph.com 2022-12-05T10:12:21.130 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.15.136 smithi136.front.sepia.ceph.com smithi136 2022-12-05T10:12:21.142 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143 smithi143.front.sepia.ceph.com smithi143 2022-12-05T10:12:21.494 INFO:teuthology.provision.fog.smithi136:Deploy complete! 2022-12-05T10:12:21.497 INFO:teuthology.provision.fog.smithi143:Deploy complete! 2022-12-05T10:12:21.687 INFO:teuthology.lock.ops:Checking smithi136.front.sepia.ceph.com 2022-12-05T10:12:21.688 INFO:teuthology.lock.ops:New key found. Updating... 2022-12-05T10:12:21.711 INFO:teuthology.lock.ops:Checking smithi143.front.sepia.ceph.com 2022-12-05T10:12:21.711 INFO:teuthology.lock.ops:New key found. Updating... 2022-12-05T10:12:21.730 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: set os type and version on server 2022-12-05T10:12:21.731 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2022-12-05T10:12:21.941 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2022-12-05T10:12:21.998 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2022-12-05T10:12:21.999 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="8" 2022-12-05T10:12:21.999 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2022-12-05T10:12:22.000 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2022-12-05T10:12:22.000 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="8" 2022-12-05T10:12:22.000 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el8" 2022-12-05T10:12:22.001 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 8" 2022-12-05T10:12:22.001 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2022-12-05T10:12:22.001 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-12-05T10:12:22.002 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2022-12-05T10:12:22.002 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-12-05T10:12:22.002 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-12-05T10:12:22.003 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-12-05T10:12:22.005 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2022-12-05T10:12:22.026 INFO:teuthology.lock.ops:Updating [smithi143.front.sepia.ceph.com]: set os type and version on server 2022-12-05T10:12:22.027 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2022-12-05T10:12:22.251 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2022-12-05T10:12:22.309 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="CentOS Stream" 2022-12-05T10:12:22.309 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="8" 2022-12-05T10:12:22.309 INFO:teuthology.orchestra.run.smithi143.stdout:ID="centos" 2022-12-05T10:12:22.310 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="rhel fedora" 2022-12-05T10:12:22.310 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="8" 2022-12-05T10:12:22.310 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el8" 2022-12-05T10:12:22.311 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="CentOS Stream 8" 2022-12-05T10:12:22.311 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31" 2022-12-05T10:12:22.311 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-12-05T10:12:22.312 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://centos.org/" 2022-12-05T10:12:22.312 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-12-05T10:12:22.313 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-12-05T10:12:22.313 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-12-05T10:12:22.314 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2022-12-05T10:12:22.449 INFO:teuthology.dispatcher.supervisor:Running job 7103339 2022-12-05T10:12:22.450 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_4da97cf64e542f347ec47b7bdbe5eca99759f9b7/virtualenv/bin/teuthology -v --owner scheduled_dis@teuthology --archive /home/teuthworker/archive/dis-2022-12-05_08:59:55-rbd-wip-dis-testing-distro-default-smithi/7103339 --name dis-2022-12-05_08:59:55-rbd-wip-dis-testing-distro-default-smithi --description rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} -- /home/teuthworker/archive/dis-2022-12-05_08:59:55-rbd-wip-dis-testing-distro-default-smithi/7103339/orig.config.yaml 2022-12-05T10:12:22.455 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/dis-2022-12-05_08:59:55-rbd-wip-dis-testing-distro-default-smithi/7103339 2022-12-05T10:12:22.456 INFO:teuthology.dispatcher.supervisor:Job PID: 1643 2022-12-05T10:12:22.456 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-12-05T22:13:18.209 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2022-12-05T22:13:18.386 INFO:teuthology.kill:Killing Pids: {1643} 2022-12-05T22:13:19.922 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'] 2022-12-05T22:13:19.923 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6'] 2022-12-05T22:13:19.923 INFO:teuthology.misc:Compressing logs... 2022-12-05T22:13:19.924 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2022-12-05T22:13:20.135 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2022-12-05T22:13:20.529 INFO:teuthology.misc:Compressing logs... 2022-12-05T22:14:01.820 INFO:teuthology.misc:Compressing logs... 2022-12-05T22:17:21.082 INFO:teuthology.kill:No teuthology processes running 2022-12-05T22:17:22.519 INFO:teuthology.kill:Nuking machines: ['smithi136', 'smithi143'] 2022-12-05T22:17:23.715 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_teuthology_4da97cf64e542f347ec47b7bdbe5eca99759f9b7/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release. 2022-12-05T22:17:23.716 INFO:teuthology.kill: from cryptography.hazmat.backends import default_backend 2022-12-05T22:17:23.716 INFO:teuthology.kill:2022-12-05 22:17:23,714.714 INFO:teuthology.nuke:targets: 2022-12-05T22:17:23.716 INFO:teuthology.kill: smithi136.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/XXpHPH34Kui7ohtrK8noNLXOix7G8tJ8dxsjnA6zlF5tA/FjROQnz4MFdFRmgdvR7V3Rt144QEST+xqYoReaNlb/vp5T4y3h/9JV5MWq9whyxuKHuareA1KGAx7CRDyHSk9gU+Glv7GWH5gGl66qwMj8Pk1J8p1fb8wnD1RTa2bsn9uQwKgzNlhz0zp9P0Z56gZPMKew7oTir2sh9/AYeoF4oBSUW6mE0QyGxcF63WZcbp0OwyNT52epay1i7UeS+DPUSxdrYKDmTho/fEz/8nmfmHTbJofaRRiiAH2KQlLENjGxCfkKbwitRDlflTursQLMoO7rghWYwLtIJrAHnyqYWSBX2AhqqBQBQvqYXjPA9xSvnnXLgstqkRhh5447x6Assns0QN1CjByGRxCO35Tphms3Al92ARG/GOOv58PqPd0gW9z96ByJpw7ohYXTREd9/mYPXCrkRkNRxkSSTa2vbSJj0Ejg37OnrkiRD9MU+A+pxOmRqMZhbm2zEu8= 2022-12-05T22:17:23.717 INFO:teuthology.kill: smithi143.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdWF0z2kTil7qFmbNTURw7ju407BGyN3A32u3+U815nOFtSfsZbqgctYP68CKssuGkdA5Ch17QKQGOJzVZ430Fp9gT3JlwQXXn+YVh1zoJqkYmrKLSMTbPs3l0hVkwmnE7Kx9OEQ7GGEDPN5HPjmyUQPIaCLGtlseg9mvpxiIHxz1gHXGjB4XvZMRvJHAH/keMfbBlXej3/henRdNv2x019LLRNtjSLA9fY4jALaSZdwEBQfEQbrVxytoG4JHXk1d8fDLsH4gqon2ZYF5e5u36orBB/PG02YhB4NIc6PFWIuW0Nm/6yVs4ebrC2O9FFYhnHMYlkPyg9lvRpuOiHY/4MuUNL1yabpzQ1zhrowRQj6COgbMMHpaagFiJoELzxmcIBwasg7UMJAJxjcFhyQ/RFjJQM49Fzi5mcxREl4+zNS5ODEDP629bDXDbeq5K47Z9FhQ22G9A35NN+6VW6HpH8bnefIT75SHOUqRXZJkgXQcrHMYF3mu9/E/ixmBXXPs= 2022-12-05T22:17:23.717 INFO:teuthology.kill:2022-12-05 22:17:23,715.715 INFO:teuthology.nuke:Checking targets against current locks 2022-12-05T22:17:23.771 INFO:teuthology.kill:2022-12-05 22:17:23,770.770 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-12-05T22:17:23.794 INFO:teuthology.kill:2022-12-05 22:17:23,794.794 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-12-05T22:17:23.867 INFO:teuthology.kill:2022-12-05 22:17:23,867.867 INFO:teuthology.orchestra.console:Power off smithi136 2022-12-05T22:17:23.914 INFO:teuthology.kill:2022-12-05 22:17:23,914.914 INFO:teuthology.orchestra.console:Power off smithi143 2022-12-05T22:17:36.365 INFO:teuthology.kill:2022-12-05 22:17:36,364.364 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2022-12-05T22:17:36.399 INFO:teuthology.kill:2022-12-05 22:17:36,399.399 INFO:teuthology.orchestra.console:Power off for smithi143 completed 2022-12-05T22:17:36.535 INFO:teuthology.kill:2022-12-05 22:17:36,534.534 INFO:teuthology.lock.ops:unlocked: smithi136.front.sepia.ceph.com 2022-12-05T22:17:36.568 INFO:teuthology.kill:2022-12-05 22:17:36,568.568 INFO:teuthology.lock.ops:unlocked: smithi143.front.sepia.ceph.com 2022-12-05T22:19:36.971 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-12-05T22:19:37.095 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi143 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-12-05_16:16:42-fs-wip-vshankar-testing-20221130.043104-testing-default-smithi/7103644