2022-02-10T07:56:56.737 INFO:root:teuthology version: 1.1.0-7cebb3f2 2022-02-10T07:56:56.869 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi085.front.sepia.ceph.com' reimaging 2022-02-10T07:56:56.870 INFO:teuthology.lock.ops:Updating [ubuntu@smithi085.front.sepia.ceph.com]: reset os type and version on server 2022-02-10T07:56:56.870 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server 2022-02-10T07:56:56.890 INFO:teuthology.lock.ops:Node 'ubuntu@smithi085.front.sepia.ceph.com' reimaging is complete 2022-02-10T07:56:56.891 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi121.front.sepia.ceph.com' reimaging 2022-02-10T07:56:56.891 INFO:teuthology.lock.ops:Updating [ubuntu@smithi121.front.sepia.ceph.com]: reset os type and version on server 2022-02-10T07:56:56.892 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2022-02-10T07:56:56.920 INFO:teuthology.lock.ops:Node 'ubuntu@smithi121.front.sepia.ceph.com' reimaging is complete 2022-02-10T07:56:57.118 INFO:teuthology.provision.fog.smithi085:Scheduling deploy of centos 8.stream 2022-02-10T07:56:57.172 INFO:teuthology.provision.fog.smithi121:Scheduling deploy of centos 8.stream 2022-02-10T07:56:57.683 INFO:teuthology.orchestra.console:Power off smithi085 2022-02-10T07:56:57.773 INFO:teuthology.orchestra.console:Power off smithi121 2022-02-10T07:56:57.822 INFO:teuthology.orchestra.console:Power off for smithi085 completed 2022-02-10T07:56:57.911 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2022-02-10T07:56:57.924 INFO:teuthology.orchestra.console:Power on smithi085 2022-02-10T07:56:58.012 INFO:teuthology.orchestra.console:Power on smithi121 2022-02-10T07:57:06.290 INFO:teuthology.orchestra.console:Power on for smithi085 completed 2022-02-10T07:57:06.368 INFO:teuthology.orchestra.console:Power on for smithi121 completed 2022-02-10T07:57:06.392 INFO:teuthology.provision.fog.smithi085:Waiting for deploy to finish 2022-02-10T07:57:06.469 INFO:teuthology.provision.fog.smithi121:Waiting for deploy to finish 2022-02-10T08:01:04.757 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-10T08:01:14.012 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-10T08:01:30.235 INFO:teuthology.provision.fog.smithi121:Node is ready 2022-02-10T08:01:30.258 INFO:teuthology.orchestra.run.smithi121.stdout:smithi121.front.sepia.ceph.com 2022-02-10T08:01:30.319 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.15.121 smithi121.front.sepia.ceph.com smithi121 2022-02-10T08:01:30.688 INFO:teuthology.provision.fog.smithi121:Deploy complete! 2022-02-10T08:01:39.079 INFO:teuthology.provision.fog.smithi085:Node is ready 2022-02-10T08:01:39.101 INFO:teuthology.orchestra.run.smithi085.stdout:smithi085.front.sepia.ceph.com 2022-02-10T08:01:39.161 INFO:teuthology.orchestra.run.smithi085.stdout:172.21.15.85 smithi085.front.sepia.ceph.com smithi085 2022-02-10T08:01:39.520 INFO:teuthology.provision.fog.smithi085:Deploy complete! 2022-02-10T08:01:39.678 INFO:teuthology.lock.ops:Checking smithi085.front.sepia.ceph.com 2022-02-10T08:01:39.679 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-10T08:01:39.698 INFO:teuthology.lock.ops:Checking smithi121.front.sepia.ceph.com 2022-02-10T08:01:39.698 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-10T08:01:39.719 INFO:teuthology.lock.ops:Updating [smithi085.front.sepia.ceph.com]: set os type and version on server 2022-02-10T08:01:39.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi085.front.sepia.ceph.com' 2022-02-10T08:01:39.916 INFO:teuthology.orchestra.run.smithi085.stdout:x86_64 2022-02-10T08:01:39.973 INFO:teuthology.orchestra.run.smithi085.stdout:NAME="CentOS Stream" 2022-02-10T08:01:39.974 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION="8" 2022-02-10T08:01:39.974 INFO:teuthology.orchestra.run.smithi085.stdout:ID="centos" 2022-02-10T08:01:39.974 INFO:teuthology.orchestra.run.smithi085.stdout:ID_LIKE="rhel fedora" 2022-02-10T08:01:39.975 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION_ID="8" 2022-02-10T08:01:39.975 INFO:teuthology.orchestra.run.smithi085.stdout:PLATFORM_ID="platform:el8" 2022-02-10T08:01:39.975 INFO:teuthology.orchestra.run.smithi085.stdout:PRETTY_NAME="CentOS Stream 8" 2022-02-10T08:01:39.975 INFO:teuthology.orchestra.run.smithi085.stdout:ANSI_COLOR="0;31" 2022-02-10T08:01:39.976 INFO:teuthology.orchestra.run.smithi085.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-02-10T08:01:39.976 INFO:teuthology.orchestra.run.smithi085.stdout:HOME_URL="https://centos.org/" 2022-02-10T08:01:39.976 INFO:teuthology.orchestra.run.smithi085.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-10T08:01:39.976 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-10T08:01:39.976 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-02-10T08:01:39.978 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server 2022-02-10T08:01:39.995 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: set os type and version on server 2022-02-10T08:01:39.996 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2022-02-10T08:01:40.191 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2022-02-10T08:01:40.250 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2022-02-10T08:01:40.250 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="8" 2022-02-10T08:01:40.251 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2022-02-10T08:01:40.251 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2022-02-10T08:01:40.251 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="8" 2022-02-10T08:01:40.252 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el8" 2022-02-10T08:01:40.252 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 8" 2022-02-10T08:01:40.252 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2022-02-10T08:01:40.252 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-02-10T08:01:40.252 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2022-02-10T08:01:40.253 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-10T08:01:40.253 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-10T08:01:40.253 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-02-10T08:01:40.254 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2022-02-10T08:01:40.359 INFO:teuthology.dispatcher.supervisor:Running job 6673148 2022-02-10T08:01:40.359 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology -v --owner scheduled_amathuri@teuthology --archive /home/teuthworker/archive/amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi/6673148 --name amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} -- /home/teuthworker/archive/amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi/6673148/orig.config.yaml 2022-02-10T08:01:40.363 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi/6673148 2022-02-10T08:01:40.364 INFO:teuthology.dispatcher.supervisor:Job PID: 25351 2022-02-10T08:01:40.364 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-10T14:32:09.411 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-10T14:32:09.665 INFO:teuthology.kill:Killing Pids: {25351} 2022-02-10T14:32:10.944 INFO:teuthology.task.internal:roles: ubuntu@smithi085.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-02-10T14:32:10.945 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-02-10T14:32:10.945 INFO:teuthology.misc:Compressing logs... 2022-02-10T14:32:10.946 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi085.front.sepia.ceph.com' 2022-02-10T14:32:11.132 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2022-02-10T14:32:11.504 INFO:teuthology.misc:Compressing logs... 2022-02-10T14:32:11.860 INFO:teuthology.orchestra.run.smithi085.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-02-10T14:32:11.864 INFO:teuthology.orchestra.run.smithi121.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-02-10T14:32:12.110 INFO:teuthology.misc:Compressing logs... 2022-02-10T14:32:49.579 INFO:teuthology.kill:No teuthology processes running 2022-02-10T14:32:50.722 INFO:teuthology.kill:Nuking machines: ['smithi085', 'smithi121'] 2022-02-10T14:32:51.683 INFO:teuthology.kill:2022-02-10 14:32:51,683.683 INFO:teuthology.nuke:targets: 2022-02-10T14:32:51.684 INFO:teuthology.kill: smithi085.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDb8Lxu8e+W4QfVKHrPVCrZWN4ceBsekcB3dOjhOiWOuMm2+TmSggM+hB35Hi00AoQxQsy/rGe3OEIltdW51cDtKmNCclm4e84JX27TKWK8v00MhqJqSXIv3gZrc5xKbOo4Q2FfPzi17rxhR16EUASP3b0kfB3/s7t3rkMMl9O3JPn2NjEZhQz3WukeG52y+TJblhGvDDktJ7R9Yzf6Kqu6xelKuJCZ75Sl156X901YkgsZJ/EPbjwl+bTE5/3Ejb5g75Gx3cCBRBr/q9STtmPThuONAmJva2Y/QLHSND3tnEBoltYaxWH0B90IJIsAI6gcxuaBCZwf6cm2Z02KLQbZnTOlM2Jn7AuAD3KrQ5rqlJBY4MPaqhdOqHkeZbvWJaxMSF6iGeSpdiW10POPcaAWUR4U42SGEIPmb9Ryf9OZyqzJondR0Tfk6KexInjmLOfBHuVBJfBa4HoueSrvif6qC+Iq4Jdt0GQsn45CWSC14r+F299IZvxbbPufOHOSNJM= 2022-02-10T14:32:51.684 INFO:teuthology.kill: smithi121.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJs/Mmby9Bvim3x50tv66rV1A42Auv7Gt6Qtyq/LB6Iv8ggBFdPaZv9rl8G3vkCM1gsuANBtRgPwiknY3cCTtO7623iihPKT/06n1E7zRfRcsA8EEJSfYjLpDecFX0E80txD0hF1z7fCCrIDZwsfhfZnVu7+sHVJMH2C0smc5G88i5UoZWrbK2gekwpxyoisYTaZXddnN20E4pSQkGJJDxwvMGWIj723qsVUMyNCKaVx9GjPHNi3UdyOyYV4utEpfUJnSAg41edNspEGNprfQJbwCVwQeF6zhrgr6QsMC88mbVErO/ZnP4MN5Y63e2bKevQZHcH4vbwvWT/bz2hHJhz6pYFF3JgS/+ZIMFXphKh6rKQ4DcMpIEcy3buTPx/qDDfEoQlMEaFfc4HJ8tX25CWusgn1Djo08ifHnxuDLvYFkfmqN5SVhl6fDA2Ajz7Tg/GcrsWbHjAxT+wXFq9zyIs2FgH68r6BqO5aumKM2AvV4aMT+0AKvuHizrOyM+ejM= 2022-02-10T14:32:51.710 INFO:teuthology.kill:2022-02-10 14:32:51,710.710 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-10T14:32:51.714 INFO:teuthology.kill:2022-02-10 14:32:51,714.714 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-10T14:32:51.793 INFO:teuthology.kill:2022-02-10 14:32:51,793.793 INFO:teuthology.orchestra.console:Power off smithi121 2022-02-10T14:32:51.807 INFO:teuthology.kill:2022-02-10 14:32:51,806.806 INFO:teuthology.orchestra.console:Power off smithi085 2022-02-10T14:33:00.163 INFO:teuthology.kill:2022-02-10 14:33:00,162.162 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2022-02-10T14:33:00.309 INFO:teuthology.kill:2022-02-10 14:33:00,309.309 INFO:teuthology.lock.ops:unlocked: smithi121.front.sepia.ceph.com 2022-02-10T14:33:04.280 INFO:teuthology.kill:2022-02-10 14:33:04,280.280 INFO:teuthology.orchestra.console:Power off for smithi085 completed 2022-02-10T14:33:04.428 INFO:teuthology.kill:2022-02-10 14:33:04,428.428 INFO:teuthology.lock.ops:unlocked: smithi085.front.sepia.ceph.com 2022-02-10T14:35:04.773 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-10T14:35:04.881 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi085 but it was locked by another job: /home/teuthworker/archive/matan-2022-02-10_14:11:48-rados:thrash-master-distro-basic-smithi/6673700 2022-02-10T14:35:04.881 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi121 but it was locked by another job: /home/teuthworker/archive/matan-2022-02-10_14:11:48-rados:thrash-master-distro-basic-smithi/6673700