2022-02-10T07:56:46.278 INFO:root:teuthology version: 1.1.0-7cebb3f2 2022-02-10T07:56:46.398 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi132.front.sepia.ceph.com' reimaging 2022-02-10T07:56:46.399 INFO:teuthology.lock.ops:Updating [ubuntu@smithi132.front.sepia.ceph.com]: reset os type and version on server 2022-02-10T07:56:46.400 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2022-02-10T07:56:46.418 INFO:teuthology.lock.ops:Node 'ubuntu@smithi132.front.sepia.ceph.com' reimaging is complete 2022-02-10T07:56:46.418 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi140.front.sepia.ceph.com' reimaging 2022-02-10T07:56:46.419 INFO:teuthology.lock.ops:Updating [ubuntu@smithi140.front.sepia.ceph.com]: reset os type and version on server 2022-02-10T07:56:46.419 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2022-02-10T07:56:46.438 INFO:teuthology.lock.ops:Node 'ubuntu@smithi140.front.sepia.ceph.com' reimaging is complete 2022-02-10T07:56:46.649 INFO:teuthology.provision.fog.smithi132:Scheduling deploy of centos 8.stream 2022-02-10T07:56:46.669 INFO:teuthology.provision.fog.smithi140:Scheduling deploy of centos 8.stream 2022-02-10T07:56:47.259 INFO:teuthology.orchestra.console:Power off smithi140 2022-02-10T07:56:47.326 INFO:teuthology.orchestra.console:Power off smithi132 2022-02-10T07:56:55.634 INFO:teuthology.orchestra.console:Power off for smithi140 completed 2022-02-10T07:56:55.687 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2022-02-10T07:56:55.736 INFO:teuthology.orchestra.console:Power on smithi140 2022-02-10T07:56:55.789 INFO:teuthology.orchestra.console:Power on smithi132 2022-02-10T07:56:59.996 INFO:teuthology.orchestra.console:Power on for smithi140 completed 2022-02-10T07:57:00.046 INFO:teuthology.orchestra.console:Power on for smithi132 completed 2022-02-10T07:57:00.098 INFO:teuthology.provision.fog.smithi140:Waiting for deploy to finish 2022-02-10T07:57:00.147 INFO:teuthology.provision.fog.smithi132:Waiting for deploy to finish 2022-02-10T08:01:31.897 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-10T08:01:33.244 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-10T08:01:33.508 INFO:teuthology.provision.fog.smithi132:Node is ready 2022-02-10T08:01:33.526 INFO:teuthology.orchestra.run.smithi132.stdout:smithi132.front.sepia.ceph.com 2022-02-10T08:01:33.585 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132 smithi132.front.sepia.ceph.com smithi132 2022-02-10T08:01:33.939 INFO:teuthology.provision.fog.smithi132:Deploy complete! 2022-02-10T08:01:57.187 INFO:teuthology.provision.fog.smithi140:Node is ready 2022-02-10T08:01:57.214 INFO:teuthology.orchestra.run.smithi140.stdout:smithi140.front.sepia.ceph.com 2022-02-10T08:01:57.276 INFO:teuthology.orchestra.run.smithi140.stdout:172.21.15.140 smithi140.front.sepia.ceph.com smithi140 2022-02-10T08:01:57.659 INFO:teuthology.provision.fog.smithi140:Deploy complete! 2022-02-10T08:01:57.825 INFO:teuthology.lock.ops:Checking smithi132.front.sepia.ceph.com 2022-02-10T08:01:57.826 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-10T08:01:57.849 INFO:teuthology.lock.ops:Checking smithi140.front.sepia.ceph.com 2022-02-10T08:01:57.849 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-10T08:01:57.870 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: set os type and version on server 2022-02-10T08:01:57.870 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2022-02-10T08:01:58.113 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2022-02-10T08:01:58.173 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Stream" 2022-02-10T08:01:58.173 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="8" 2022-02-10T08:01:58.174 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos" 2022-02-10T08:01:58.174 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora" 2022-02-10T08:01:58.174 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="8" 2022-02-10T08:01:58.174 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el8" 2022-02-10T08:01:58.175 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Stream 8" 2022-02-10T08:01:58.175 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2022-02-10T08:01:58.175 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-02-10T08:01:58.175 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://centos.org/" 2022-02-10T08:01:58.176 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-10T08:01:58.176 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-10T08:01:58.176 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-02-10T08:01:58.177 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2022-02-10T08:01:58.201 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: set os type and version on server 2022-02-10T08:01:58.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi140.front.sepia.ceph.com' 2022-02-10T08:01:58.398 INFO:teuthology.orchestra.run.smithi140.stdout:x86_64 2022-02-10T08:01:58.460 INFO:teuthology.orchestra.run.smithi140.stdout:NAME="CentOS Stream" 2022-02-10T08:01:58.460 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION="8" 2022-02-10T08:01:58.461 INFO:teuthology.orchestra.run.smithi140.stdout:ID="centos" 2022-02-10T08:01:58.461 INFO:teuthology.orchestra.run.smithi140.stdout:ID_LIKE="rhel fedora" 2022-02-10T08:01:58.461 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION_ID="8" 2022-02-10T08:01:58.461 INFO:teuthology.orchestra.run.smithi140.stdout:PLATFORM_ID="platform:el8" 2022-02-10T08:01:58.462 INFO:teuthology.orchestra.run.smithi140.stdout:PRETTY_NAME="CentOS Stream 8" 2022-02-10T08:01:58.462 INFO:teuthology.orchestra.run.smithi140.stdout:ANSI_COLOR="0;31" 2022-02-10T08:01:58.462 INFO:teuthology.orchestra.run.smithi140.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-02-10T08:01:58.462 INFO:teuthology.orchestra.run.smithi140.stdout:HOME_URL="https://centos.org/" 2022-02-10T08:01:58.463 INFO:teuthology.orchestra.run.smithi140.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-10T08:01:58.463 INFO:teuthology.orchestra.run.smithi140.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-10T08:01:58.463 INFO:teuthology.orchestra.run.smithi140.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-02-10T08:01:58.464 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2022-02-10T08:01:58.573 INFO:teuthology.dispatcher.supervisor:Running job 6673147 2022-02-10T08:01:58.574 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology -v --owner scheduled_amathuri@teuthology --archive /home/teuthworker/archive/amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi/6673147 --name amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} -- /home/teuthworker/archive/amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi/6673147/orig.config.yaml 2022-02-10T08:01:58.577 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi/6673147 2022-02-10T08:01:58.579 INFO:teuthology.dispatcher.supervisor:Job PID: 27887 2022-02-10T08:01:58.579 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-10T14:32:27.343 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-10T14:32:27.554 INFO:teuthology.kill:Killing Pids: {27887} 2022-02-10T14:32:28.772 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-02-10T14:32:28.773 INFO:teuthology.task.internal:roles: ubuntu@smithi140.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-02-10T14:32:28.773 INFO:teuthology.misc:Compressing logs... 2022-02-10T14:32:28.774 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2022-02-10T14:32:28.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi140.front.sepia.ceph.com' 2022-02-10T14:32:29.296 INFO:teuthology.misc:Compressing logs... 2022-02-10T14:32:29.655 INFO:teuthology.orchestra.run.smithi140.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-02-10T14:32:29.921 INFO:teuthology.misc:Compressing logs... 2022-02-10T14:33:10.147 INFO:teuthology.kill:No teuthology processes running 2022-02-10T14:33:11.369 INFO:teuthology.kill:Nuking machines: ['smithi132', 'smithi140'] 2022-02-10T14:33:12.342 INFO:teuthology.kill:2022-02-10 14:33:12,342.342 INFO:teuthology.nuke:targets: 2022-02-10T14:33:12.343 INFO:teuthology.kill: smithi132.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCr18cxRaBsKmP46h/aU7zJQl6F5HUc0PHZEogCtqSOgMUMVWIG9ohR+iQDidsEj39WFfjY7zZsg91uZSsQa6YFwoehj9gV0HJrWcxFhVvlhZShOpvEKRxGzuvU/gthIM+S9wruY1NrmReHmg4+D72PAO9dV/uqWzrYpkR2D3ue1w+pxxP8Y2Y3MBEKvdQSLi9stu9PkFqNN0ArypaXillFmDnmYbgNt4S4mXYd6KGl/U4wntUZJV0gOBthFcrO4FnLh9v3KqVAAbQofWvJQ6fmI6anfXqxc+ZL1WjFMuvoqw/2yZd5dU0I53r7U9MCjwGtaRMh68uyRO0tLt5eJdJSZjS1v/huEv/y1QtH8orXl86BwM9K42kYWhlydmlg9y4JisyAgE+B830QxuwH0iKPd+hUBdIiaeqrSF4n/Hf9y2Ox7fb/JnftG15J7T3C+SlGJQ47uJVG/rkHJiZ1ALs/8N4txVIowPn+Nu/MWiatv4mcWFoKEGbiAOdZs6lv8os= 2022-02-10T14:33:12.343 INFO:teuthology.kill: smithi140.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDa1JzGD/c8TRu3cQy8aQsGJl8lobMiQbt/ESx/Fank5mF5SAROSQ1TxZjfp2n6wIXP8pkwrNSHdb87MUHxs925Ez/14RnlSZ5NX46YMeiXMH6nMxEP8Lr9Tlz5V/ARr1SmURxHfJL0FMeO+mep2unaEkYdlGGt+6cPSKlJkQ7CqB2zt/DkDO1MV8ef9Xnw3qVWLj2QwgNSuEGnVAEvcvo7PavVcyOP7EssrFlLLyPjtY7ySTL5BS43fxxXae4hVvKpZWoYsXBUIb2Zl1xBfb/WKl6ZLyHzakTPMrs/vz+yLga6HhwS9OeoduYBBKIY6JFAokmxwM9yOqyyTJ0lchq25ZSn9rXhQ8UN9G2B4uDXhiFAY8+h6f2EXIXs0Q+zwA3OmFNzHaGwlVAeJP9HIAbzAgM6UNIJH1QMXTRZN2zEvO8EoHj2vbt3E+rmAEnpqZYFL+rAUaOSqW9dFPyKoOGrNL0QAFmmn8ZwJqDS+bPq9a7dmDgOuBODvxXJWJZoST8= 2022-02-10T14:33:12.370 INFO:teuthology.kill:2022-02-10 14:33:12,369.369 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-10T14:33:12.374 INFO:teuthology.kill:2022-02-10 14:33:12,374.374 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-10T14:33:12.457 INFO:teuthology.kill:2022-02-10 14:33:12,457.457 INFO:teuthology.orchestra.console:Power off smithi140 2022-02-10T14:33:12.473 INFO:teuthology.kill:2022-02-10 14:33:12,472.472 INFO:teuthology.orchestra.console:Power off smithi132 2022-02-10T14:33:20.830 INFO:teuthology.kill:2022-02-10 14:33:20,829.829 INFO:teuthology.orchestra.console:Power off for smithi140 completed 2022-02-10T14:33:20.836 INFO:teuthology.kill:2022-02-10 14:33:20,836.836 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2022-02-10T14:33:20.973 INFO:teuthology.kill:2022-02-10 14:33:20,973.973 INFO:teuthology.lock.ops:unlocked: smithi140.front.sepia.ceph.com 2022-02-10T14:33:20.977 INFO:teuthology.kill:2022-02-10 14:33:20,977.977 INFO:teuthology.lock.ops:unlocked: smithi132.front.sepia.ceph.com 2022-02-10T14:35:21.297 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-10T14:35:21.414 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi132 but it was locked by another job: /home/teuthworker/archive/matan-2022-02-10_14:11:48-rados:thrash-master-distro-basic-smithi/6673701 2022-02-10T14:35:21.414 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi140 but it was locked by another job: /home/teuthworker/archive/matan-2022-02-10_14:11:48-rados:thrash-master-distro-basic-smithi/6673702