2022-03-24T02:42:51.740 INFO:root:teuthology version: 1.1.0-a4e18662 2022-03-24T02:42:51.859 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi121.front.sepia.ceph.com' reimaging 2022-03-24T02:42:51.860 INFO:teuthology.lock.ops:Updating [ubuntu@smithi121.front.sepia.ceph.com]: reset os type and version on server 2022-03-24T02:42:51.860 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2022-03-24T02:42:51.878 INFO:teuthology.lock.ops:Node 'ubuntu@smithi121.front.sepia.ceph.com' reimaging is complete 2022-03-24T02:42:51.879 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi156.front.sepia.ceph.com' reimaging 2022-03-24T02:42:51.879 INFO:teuthology.lock.ops:Updating [ubuntu@smithi156.front.sepia.ceph.com]: reset os type and version on server 2022-03-24T02:42:51.879 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2022-03-24T02:42:51.896 INFO:teuthology.lock.ops:Node 'ubuntu@smithi156.front.sepia.ceph.com' reimaging is complete 2022-03-24T02:42:52.106 INFO:teuthology.provision.fog.smithi156:Scheduling deploy of rhel 8.4 2022-03-24T02:42:52.112 INFO:teuthology.provision.fog.smithi121:Scheduling deploy of rhel 8.4 2022-03-24T02:42:52.521 INFO:teuthology.orchestra.console:Power off smithi156 2022-03-24T02:42:52.610 INFO:teuthology.orchestra.console:Power off smithi121 2022-03-24T02:43:00.973 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2022-03-24T02:43:01.075 INFO:teuthology.orchestra.console:Power on smithi121 2022-03-24T02:43:05.011 INFO:teuthology.orchestra.console:Power off for smithi156 completed 2022-03-24T02:43:05.112 INFO:teuthology.orchestra.console:Power on smithi156 2022-03-24T02:43:09.366 INFO:teuthology.orchestra.console:Power on for smithi156 completed 2022-03-24T02:43:09.435 INFO:teuthology.orchestra.console:Power on for smithi121 completed 2022-03-24T02:43:09.468 INFO:teuthology.provision.fog.smithi156:Waiting for deploy to finish 2022-03-24T02:43:09.536 INFO:teuthology.provision.fog.smithi121:Waiting for deploy to finish 2022-03-24T02:47:28.320 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-24T02:47:38.549 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-24T02:47:38.792 INFO:teuthology.provision.fog.smithi121:Node is ready 2022-03-24T02:47:38.848 INFO:teuthology.orchestra.run.smithi121.stdout:smithi121.front.sepia.ceph.com 2022-03-24T02:47:38.904 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.15.121 smithi121.front.sepia.ceph.com smithi121 2022-03-24T02:47:39.223 INFO:teuthology.provision.fog.smithi121:Deploy complete! 2022-03-24T02:47:48.479 INFO:teuthology.provision.fog.smithi156:Node is ready 2022-03-24T02:47:48.498 INFO:teuthology.orchestra.run.smithi156.stdout:smithi156.front.sepia.ceph.com 2022-03-24T02:47:48.554 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156 smithi156.front.sepia.ceph.com smithi156 2022-03-24T02:47:48.913 INFO:teuthology.provision.fog.smithi156:Deploy complete! 2022-03-24T02:47:49.101 INFO:teuthology.lock.ops:Checking smithi121.front.sepia.ceph.com 2022-03-24T02:47:49.102 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-24T02:47:49.123 INFO:teuthology.lock.ops:Checking smithi156.front.sepia.ceph.com 2022-03-24T02:47:49.124 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-24T02:47:49.141 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: set os type and version on server 2022-03-24T02:47:49.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2022-03-24T02:47:49.373 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2022-03-24T02:47:49.427 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="Red Hat Enterprise Linux" 2022-03-24T02:47:49.428 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="8.4 (Ootpa)" 2022-03-24T02:47:49.428 INFO:teuthology.orchestra.run.smithi121.stdout:ID="rhel" 2022-03-24T02:47:49.428 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="fedora" 2022-03-24T02:47:49.429 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="8.4" 2022-03-24T02:47:49.429 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el8" 2022-03-24T02:47:49.429 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-03-24T02:47:49.430 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2022-03-24T02:47:49.430 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-03-24T02:47:49.430 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://www.redhat.com/" 2022-03-24T02:47:49.430 INFO:teuthology.orchestra.run.smithi121.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-03-24T02:47:49.431 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-24T02:47:49.431 INFO:teuthology.orchestra.run.smithi121.stdout: 2022-03-24T02:47:49.431 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-24T02:47:49.431 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-03-24T02:47:49.432 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-03-24T02:47:49.432 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-03-24T02:47:49.433 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2022-03-24T02:47:49.451 INFO:teuthology.lock.ops:Updating [smithi156.front.sepia.ceph.com]: set os type and version on server 2022-03-24T02:47:49.451 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2022-03-24T02:47:49.679 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2022-03-24T02:47:49.733 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="Red Hat Enterprise Linux" 2022-03-24T02:47:49.733 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="8.4 (Ootpa)" 2022-03-24T02:47:49.733 INFO:teuthology.orchestra.run.smithi156.stdout:ID="rhel" 2022-03-24T02:47:49.734 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="fedora" 2022-03-24T02:47:49.734 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="8.4" 2022-03-24T02:47:49.734 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el8" 2022-03-24T02:47:49.734 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2022-03-24T02:47:49.735 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2022-03-24T02:47:49.735 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2022-03-24T02:47:49.735 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://www.redhat.com/" 2022-03-24T02:47:49.735 INFO:teuthology.orchestra.run.smithi156.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-03-24T02:47:49.735 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-24T02:47:49.736 INFO:teuthology.orchestra.run.smithi156.stdout: 2022-03-24T02:47:49.736 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-24T02:47:49.736 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2022-03-24T02:47:49.736 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-03-24T02:47:49.737 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2022-03-24T02:47:49.737 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2022-03-24T02:47:49.829 INFO:teuthology.dispatcher.supervisor:Running job 6756431 2022-03-24T02:47:49.829 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_a4e18662e5de936eb0ac089e531b933ecbcadb0a/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-03-23_20:48:04-rbd-wip-yuri6-testing-2022-03-23-1009-pacific-distro-default-smithi/6756431 --name yuriw-2022-03-23_20:48:04-rbd-wip-yuri6-testing-2022-03-23-1009-pacific-distro-default-smithi --description rbd/persistent-writeback-cache/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{rhel_8} 4-pool/big-cache 5-cache-mode/rwl 6-workloads/qemu_xfstests} -- /home/teuthworker/archive/yuriw-2022-03-23_20:48:04-rbd-wip-yuri6-testing-2022-03-23-1009-pacific-distro-default-smithi/6756431/orig.config.yaml 2022-03-24T02:47:49.833 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-03-23_20:48:04-rbd-wip-yuri6-testing-2022-03-23-1009-pacific-distro-default-smithi/6756431 2022-03-24T02:47:49.835 INFO:teuthology.dispatcher.supervisor:Job PID: 21803 2022-03-24T02:47:49.835 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-24T09:18:18.667 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-24T09:18:18.816 INFO:teuthology.kill:Killing Pids: {21803} 2022-03-24T09:18:20.029 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1'] 2022-03-24T09:18:20.030 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'osd.2', 'osd.3', 'client.0'] 2022-03-24T09:18:20.030 INFO:teuthology.misc:Compressing logs... 2022-03-24T09:18:20.031 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2022-03-24T09:18:20.260 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2022-03-24T09:18:20.916 INFO:teuthology.misc:Compressing logs... 2022-03-24T09:18:21.061 INFO:teuthology.misc:Compressing logs... 2022-03-24T09:20:19.815 INFO:teuthology.kill:No teuthology processes running 2022-03-24T09:20:20.934 INFO:teuthology.kill:Nuking machines: ['smithi121', 'smithi156'] 2022-03-24T09:20:21.885 INFO:teuthology.kill:2022-03-24 09:20:21,885.885 INFO:teuthology.nuke:targets: 2022-03-24T09:20:21.886 INFO:teuthology.kill: smithi121.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCx2CuMqwfzd4xjc4uJHPMQ7UY91NLEZFioBer5ZOUlq2Gv2XJrUdWGEKzqqtHHEC0XYe8scOl8RAclaGRh4JWLWJQLkBX56v+zJYbPcheqLmmiUNRuzRYRYpctuJHOsRr279CRGgN0Z3v7vrQRO//Qyfxq+oMEqpY6Rn9zp35KkAzcJOxS2kKtbDojEdaCOt3jBTB64+YOMz4NxOuceLScFMY89oLpAkE39RGCwr3CKx0aygM440l4OhTJupDfDhe299ro5ZOgypzeKuVrXP73UdfJmUUQou1VrtXKOKhMc1CXxrA9Vd8Qop/Rt12g1m1QtOQuCv/b/fuqiJIhl74HE6hMg/mNwhHWZzJtUxo+NFxsDtwPty/dOhTnRn1Roc7xKz5QlBebzhBlpklxq6ftQYEP5Sw+q10D+OR80ru07XQ+U0btnxm/vGdjXmcnjBExJrANyPh+DqCrUIU7gBscRcXEWPH3b6cVN2WIFiIklwOwPGQQKbHpD2NQkiovkGU= 2022-03-24T09:20:21.886 INFO:teuthology.kill: smithi156.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2eCl/JXNRLUgg+MB4SC0sfzGpMd0uvHHB3AVTG+RlfK+bhH/KYitVNTffIZkfPSv7JxDo1xMymYN2gby58uh9z3Smx3jWh0KHdvv8gdeH8e7dAhwkffMLCmIYepglMpJe+QNGh0wauN4os0Qch5sYVDHK5PKvBbNoYykDRE/yYSbnMOgci2fCQda889JdkgTPZ/nLtepENELt+VQl2c9KdD1j0PKFna4c7hdGVNAamJBMrJ80uMbUgBQpRUGU1vpON9q5k/lP7OmI/yWKQfdcSpQiC2QhO+wVhrdFBCs8EDCYh2/LND5mOXWsoK8rb9waRqM4UyTneGi7Q5RxAZqe2FPP0D6saz1iumImvGhEBPj70q4V9qGue5o70mFRgbJsH7fnMT58k/1ApgKfic7tsKXwzEXzxGbcaQqWqZ3x5/6LNuRFHWHeKwMgLR+Z19sJlyx09nieccC8JTB713NbjJeTd1NKp1HzqeskYlmtOKjJXICL87amxqQA1TbHKs8= 2022-03-24T09:20:21.911 INFO:teuthology.kill:2022-03-24 09:20:21,911.911 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-24T09:20:21.914 INFO:teuthology.kill:2022-03-24 09:20:21,914.914 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-24T09:20:21.990 INFO:teuthology.kill:2022-03-24 09:20:21,990.990 INFO:teuthology.orchestra.console:Power off smithi121 2022-03-24T09:20:22.002 INFO:teuthology.kill:2022-03-24 09:20:22,002.002 INFO:teuthology.orchestra.console:Power off smithi156 2022-03-24T09:20:30.361 INFO:teuthology.kill:2022-03-24 09:20:30,360.360 INFO:teuthology.orchestra.console:Power off for smithi156 completed 2022-03-24T09:20:30.506 INFO:teuthology.kill:2022-03-24 09:20:30,506.506 INFO:teuthology.lock.ops:unlocked: smithi156.front.sepia.ceph.com 2022-03-24T09:20:34.464 INFO:teuthology.kill:2022-03-24 09:20:34,464.464 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2022-03-24T09:20:34.617 INFO:teuthology.kill:2022-03-24 09:20:34,617.617 INFO:teuthology.lock.ops:unlocked: smithi121.front.sepia.ceph.com 2022-03-24T09:22:34.947 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-24T09:22:35.027 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi121 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-23_14:01:37-rbd-wip-yuri10-testing-2022-03-22-1809-pacific-distro-default-smithi/6755772 2022-03-24T09:22:35.028 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi156 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-03-23_14:01:37-rbd-wip-yuri10-testing-2022-03-22-1809-pacific-distro-default-smithi/6755772