2021-11-23T19:38:26.717 INFO:root:teuthology version: 1.1.0-6cfddd5e 2021-11-23T19:38:26.853 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi084.front.sepia.ceph.com' reimaging 2021-11-23T19:38:26.854 INFO:teuthology.lock.ops:Updating [ubuntu@smithi084.front.sepia.ceph.com]: reset os type and version on server 2021-11-23T19:38:26.854 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2021-11-23T19:38:26.879 INFO:teuthology.lock.ops:Node 'ubuntu@smithi084.front.sepia.ceph.com' reimaging is complete 2021-11-23T19:38:26.879 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi153.front.sepia.ceph.com' reimaging 2021-11-23T19:38:26.879 INFO:teuthology.lock.ops:Updating [ubuntu@smithi153.front.sepia.ceph.com]: reset os type and version on server 2021-11-23T19:38:26.880 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2021-11-23T19:38:26.900 INFO:teuthology.lock.ops:Node 'ubuntu@smithi153.front.sepia.ceph.com' reimaging is complete 2021-11-23T19:38:27.094 INFO:teuthology.provision.fog.smithi084:Scheduling deploy of centos 8.stream 2021-11-23T19:38:27.105 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of centos 8.stream 2021-11-23T19:38:27.418 INFO:teuthology.orchestra.console:Power off smithi084 2021-11-23T19:38:27.431 INFO:teuthology.orchestra.console:Power off smithi153 2021-11-23T19:38:39.906 INFO:teuthology.orchestra.console:Power off for smithi084 completed 2021-11-23T19:38:39.911 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2021-11-23T19:38:40.008 INFO:teuthology.orchestra.console:Power on smithi084 2021-11-23T19:38:40.016 INFO:teuthology.orchestra.console:Power on smithi153 2021-11-23T19:38:44.264 INFO:teuthology.orchestra.console:Power on for smithi084 completed 2021-11-23T19:38:44.268 INFO:teuthology.orchestra.console:Power on for smithi153 completed 2021-11-23T19:38:44.366 INFO:teuthology.provision.fog.smithi084:Waiting for deploy to finish 2021-11-23T19:38:44.370 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish 2021-11-23T19:43:34.108 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-23T19:43:34.421 INFO:teuthology.provision.fog.smithi084:Node is ready 2021-11-23T19:43:34.438 INFO:teuthology.orchestra.run.smithi084.stdout:smithi084.front.sepia.ceph.com 2021-11-23T19:43:34.494 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84 smithi084.front.sepia.ceph.com smithi084 2021-11-23T19:43:34.833 INFO:teuthology.provision.fog.smithi084:Deploy complete! 2021-11-23T19:43:49.090 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-11-23T19:43:49.385 INFO:teuthology.provision.fog.smithi153:Node is ready 2021-11-23T19:43:49.402 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com 2021-11-23T19:43:49.459 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153 2021-11-23T19:43:49.802 INFO:teuthology.provision.fog.smithi153:Deploy complete! 2021-11-23T19:43:49.992 INFO:teuthology.lock.ops:Checking smithi084.front.sepia.ceph.com 2021-11-23T19:43:49.992 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-23T19:43:50.012 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com 2021-11-23T19:43:50.012 INFO:teuthology.lock.ops:New key found. Updating... 2021-11-23T19:43:50.029 INFO:teuthology.lock.ops:Updating [smithi084.front.sepia.ceph.com]: set os type and version on server 2021-11-23T19:43:50.030 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2021-11-23T19:43:50.268 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2021-11-23T19:43:50.324 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Stream" 2021-11-23T19:43:50.325 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="8" 2021-11-23T19:43:50.325 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos" 2021-11-23T19:43:50.325 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora" 2021-11-23T19:43:50.326 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="8" 2021-11-23T19:43:50.326 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el8" 2021-11-23T19:43:50.326 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Stream 8" 2021-11-23T19:43:50.326 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2021-11-23T19:43:50.327 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-11-23T19:43:50.327 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://centos.org/" 2021-11-23T19:43:50.327 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-23T19:43:50.327 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-23T19:43:50.328 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-11-23T19:43:50.329 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2021-11-23T19:43:50.347 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server 2021-11-23T19:43:50.347 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2021-11-23T19:43:50.533 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2021-11-23T19:43:50.589 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream" 2021-11-23T19:43:50.589 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8" 2021-11-23T19:43:50.589 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2021-11-23T19:43:50.590 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2021-11-23T19:43:50.590 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8" 2021-11-23T19:43:50.590 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8" 2021-11-23T19:43:50.590 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 8" 2021-11-23T19:43:50.591 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2021-11-23T19:43:50.591 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-11-23T19:43:50.591 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/" 2021-11-23T19:43:50.591 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-11-23T19:43:50.591 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-11-23T19:43:50.592 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-11-23T19:43:50.592 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2021-11-23T19:43:50.690 INFO:teuthology.dispatcher.supervisor:Running job 6522483 2021-11-23T19:43:50.691 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_6cfddd5e1c7d5ede1367cf3087c21af44eb88356/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-11-23_15:13:21-rados-wip-yuri2-testing-2021-11-22-1532-distro-basic-smithi/6522483 --name yuriw-2021-11-23_15:13:21-rados-wip-yuri2-testing-2021-11-22-1532-distro-basic-smithi --description rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8.stream} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} -- /home/teuthworker/archive/yuriw-2021-11-23_15:13:21-rados-wip-yuri2-testing-2021-11-22-1532-distro-basic-smithi/6522483/orig.config.yaml 2021-11-23T19:43:50.694 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-11-23_15:13:21-rados-wip-yuri2-testing-2021-11-22-1532-distro-basic-smithi/6522483 2021-11-23T19:43:50.695 INFO:teuthology.dispatcher.supervisor:Job PID: 3296 2021-11-23T19:43:50.696 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-11-24T07:44:46.612 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-11-24T07:44:46.823 INFO:teuthology.kill:Killing Pids: {3296} 2021-11-24T07:44:48.100 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-11-24T07:44:48.101 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-11-24T07:44:48.101 INFO:teuthology.misc:Compressing logs... 2021-11-24T07:44:48.102 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2021-11-24T07:44:48.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2021-11-24T07:44:48.682 INFO:teuthology.misc:Compressing logs... 2021-11-24T07:44:49.706 INFO:teuthology.orchestra.run.smithi153.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-11-24T07:44:50.042 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-11-24T07:44:53.135 INFO:teuthology.misc:Compressing logs... 2021-11-24T07:45:11.998 INFO:teuthology.kill:No teuthology processes running 2021-11-24T07:45:13.261 INFO:teuthology.kill:Nuking machines: ['smithi084', 'smithi153'] 2021-11-24T07:45:14.339 INFO:teuthology.kill:2021-11-24 07:45:14,339.339 INFO:teuthology.nuke:targets: 2021-11-24T07:45:14.340 INFO:teuthology.kill: smithi084.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7viLbsImIrLEwSPDu56SEZ6hl3Lj6S0hm2vpk0eAgcaZ0AW8SfsW08c/ufFQsYOiy/mn+YFqs+pZSwfvgvtrF77KqR3d9RH4rG5rPPbFYHYRxSYZ6Pu0edsaW0Z8JqFZK3dO5rb+dsA/o+7oF+CwoA4llyWEWGIKLA09Q169geoU3WPd0W41ljyfTwRi9KV/SDz0Y9wlCkcEV9Yzy0eq/X+/R8ejb+giY9h1s25n4LfyP17Wh4q6N/UIoh6p5fXXrANXD6FeUkaxK8Tgy52jKG1/dsgw1luNMlhooe6t2D2Jg8/CkvC1RG5gCxc4H2Hwj17tCO+FqzsPVEEd/fEPZad9sbfhxj/MMaDAqbP1CU4/RxyQZU4CVxNs+nxB5vCROUYyYiY/VKOzKYiJAw2pKuSpvQnHMxsBZD7FrO9h/ayDWH448IryvK0JTCgF0lRp5IrfM/XmjmknOgFWJYmEc6eJieXHI07Y5wWmJEHpkfuDrFg2BAVhBrHgYPI67mz8= 2021-11-24T07:45:14.340 INFO:teuthology.kill: smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzj2+fLZsMgyc83eXkF0hU4TD6BjJwXcymUNHbhv2032kq6P22WvgFaQMS3XBeFlHtaqlJXzPwM08/ipoTt9lKft4dlNavVGkCP44Eep536/+tZOEKLVoXebFgP06d0K1QQPtWlou+q0HizPGxw2FTkGiTGbOr9riQAIkUjHrFConp6J6i7ATaHmGt0O2RCQTlwhEbKwKG9PH1cm7hRCxO25W3/75QD7hm4337Huh3nTTakUZSlT7+DSkUPkYGG2PU4sz6K4kcOJRwyE0f1FKAalERR4Uwv9r++clxFKHRK57W+p1r8wZ9tvyFwWmDWdoj+Ebx+4ROq/ZUIgDsysBjrEPKQkI5RFukgGRDzV9gbE1vUJhf3y4hxzf0JvnA1xnbBxXZSmfZJqRQUDvrpvhHKL4CVIhHAqXuPqua1bjZduUmiBsbyjpR1nWL0wIGJeM3TepvhdlmFegSXDZbR8Vb4qKkMXpENvQkKrbi/UBYW0sfYIl8xw+T09sgORV7sTU= 2021-11-24T07:45:14.366 INFO:teuthology.kill:2021-11-24 07:45:14,366.366 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-24T07:45:14.370 INFO:teuthology.kill:2021-11-24 07:45:14,370.370 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-24T07:45:14.456 INFO:teuthology.kill:2021-11-24 07:45:14,456.456 INFO:teuthology.orchestra.console:Power off smithi153 2021-11-24T07:45:14.472 INFO:teuthology.kill:2021-11-24 07:45:14,471.471 INFO:teuthology.orchestra.console:Power off smithi084 2021-11-24T07:45:22.843 INFO:teuthology.kill:2021-11-24 07:45:22,841.841 INFO:teuthology.orchestra.console:Power off for smithi084 completed 2021-11-24T07:45:22.998 INFO:teuthology.kill:2021-11-24 07:45:22,997.997 INFO:teuthology.lock.ops:unlocked: smithi084.front.sepia.ceph.com 2021-11-24T07:45:26.951 INFO:teuthology.kill:2021-11-24 07:45:26,950.950 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2021-11-24T07:45:27.123 INFO:teuthology.kill:2021-11-24 07:45:27,123.123 INFO:teuthology.lock.ops:unlocked: smithi153.front.sepia.ceph.com 2021-11-24T07:47:27.469 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-11-24T07:47:27.577 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi084 but it was locked by another job: /home/teuthworker/archive/vshankar-2021-11-24_07:14:27-fs-wip-vshankar-testing-20211124-094330-testing-default-smithi/6523794 2021-11-24T07:47:27.578 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi153 but it was locked by another job: /home/teuthworker/archive/vshankar-2021-11-24_07:14:27-fs-wip-vshankar-testing-20211124-094330-testing-default-smithi/6523793