2022-02-10T07:57:27.415 INFO:root:teuthology version: 1.1.0-7cebb3f2 2022-02-10T07:57:27.539 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging 2022-02-10T07:57:27.541 INFO:teuthology.lock.ops:Updating [ubuntu@smithi136.front.sepia.ceph.com]: reset os type and version on server 2022-02-10T07:57:27.541 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2022-02-10T07:57:27.560 INFO:teuthology.lock.ops:Node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging is complete 2022-02-10T07:57:27.561 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi173.front.sepia.ceph.com' reimaging 2022-02-10T07:57:27.561 INFO:teuthology.lock.ops:Updating [ubuntu@smithi173.front.sepia.ceph.com]: reset os type and version on server 2022-02-10T07:57:27.561 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2022-02-10T07:57:27.580 INFO:teuthology.lock.ops:Node 'ubuntu@smithi173.front.sepia.ceph.com' reimaging is complete 2022-02-10T07:57:27.777 INFO:teuthology.provision.fog.smithi136:Scheduling deploy of centos 8.stream 2022-02-10T07:57:27.801 INFO:teuthology.provision.fog.smithi173:Scheduling deploy of centos 8.stream 2022-02-10T07:57:28.520 INFO:teuthology.orchestra.console:Power off smithi136 2022-02-10T07:57:28.560 INFO:teuthology.orchestra.console:Power off smithi173 2022-02-10T07:57:36.916 INFO:teuthology.orchestra.console:Power off for smithi173 completed 2022-02-10T07:57:37.017 INFO:teuthology.orchestra.console:Power on smithi173 2022-02-10T07:57:41.007 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2022-02-10T07:57:41.108 INFO:teuthology.orchestra.console:Power on smithi136 2022-02-10T07:57:45.493 INFO:teuthology.orchestra.console:Power on for smithi173 completed 2022-02-10T07:57:45.594 INFO:teuthology.provision.fog.smithi173:Waiting for deploy to finish 2022-02-10T07:57:49.473 INFO:teuthology.orchestra.console:Power on for smithi136 completed 2022-02-10T07:57:49.574 INFO:teuthology.provision.fog.smithi136:Waiting for deploy to finish 2022-02-10T08:01:54.813 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-10T08:01:58.678 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-10T08:02:20.260 INFO:teuthology.provision.fog.smithi173:Node is ready 2022-02-10T08:02:20.284 INFO:teuthology.orchestra.run.smithi173.stdout:smithi173.front.sepia.ceph.com 2022-02-10T08:02:20.344 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.15.173 smithi173.front.sepia.ceph.com smithi173 2022-02-10T08:02:20.706 INFO:teuthology.provision.fog.smithi173:Deploy complete! 2022-02-10T08:02:23.669 INFO:teuthology.provision.fog.smithi136:Node is ready 2022-02-10T08:02:23.691 INFO:teuthology.orchestra.run.smithi136.stdout:smithi136.front.sepia.ceph.com 2022-02-10T08:02:23.752 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.15.136 smithi136.front.sepia.ceph.com smithi136 2022-02-10T08:02:24.141 INFO:teuthology.provision.fog.smithi136:Deploy complete! 2022-02-10T08:02:24.320 INFO:teuthology.lock.ops:Checking smithi136.front.sepia.ceph.com 2022-02-10T08:02:24.321 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-10T08:02:24.345 INFO:teuthology.lock.ops:Checking smithi173.front.sepia.ceph.com 2022-02-10T08:02:24.345 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-10T08:02:24.368 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: set os type and version on server 2022-02-10T08:02:24.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2022-02-10T08:02:24.606 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2022-02-10T08:02:24.662 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2022-02-10T08:02:24.662 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="8" 2022-02-10T08:02:24.663 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2022-02-10T08:02:24.663 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2022-02-10T08:02:24.663 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="8" 2022-02-10T08:02:24.663 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el8" 2022-02-10T08:02:24.663 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 8" 2022-02-10T08:02:24.664 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2022-02-10T08:02:24.664 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-02-10T08:02:24.664 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2022-02-10T08:02:24.664 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-10T08:02:24.665 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-10T08:02:24.665 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-02-10T08:02:24.666 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2022-02-10T08:02:24.685 INFO:teuthology.lock.ops:Updating [smithi173.front.sepia.ceph.com]: set os type and version on server 2022-02-10T08:02:24.686 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2022-02-10T08:02:24.891 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2022-02-10T08:02:24.951 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2022-02-10T08:02:24.951 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="8" 2022-02-10T08:02:24.951 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2022-02-10T08:02:24.951 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2022-02-10T08:02:24.952 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="8" 2022-02-10T08:02:24.952 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el8" 2022-02-10T08:02:24.952 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 8" 2022-02-10T08:02:24.952 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2022-02-10T08:02:24.953 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-02-10T08:02:24.953 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2022-02-10T08:02:24.953 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-02-10T08:02:24.953 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-02-10T08:02:24.953 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-02-10T08:02:24.954 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2022-02-10T08:02:25.069 INFO:teuthology.dispatcher.supervisor:Running job 6673150 2022-02-10T08:02:25.069 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology -v --owner scheduled_amathuri@teuthology --archive /home/teuthworker/archive/amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi/6673150 --name amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} -- /home/teuthworker/archive/amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi/6673150/orig.config.yaml 2022-02-10T08:02:25.074 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/amathuri-2022-02-10_06:16:41-rados-master-distro-default-smithi/6673150 2022-02-10T08:02:25.075 INFO:teuthology.dispatcher.supervisor:Job PID: 583 2022-02-10T08:02:25.076 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-02-10T14:32:35.751 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-02-10T14:32:35.951 INFO:teuthology.kill:Killing Pids: {583} 2022-02-10T14:32:37.165 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-02-10T14:32:37.166 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-02-10T14:32:37.166 INFO:teuthology.misc:Compressing logs... 2022-02-10T14:32:37.167 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2022-02-10T14:32:37.353 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2022-02-10T14:32:37.658 INFO:teuthology.misc:Compressing logs... 2022-02-10T14:32:38.050 INFO:teuthology.orchestra.run.smithi173.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2022-02-10T14:32:38.453 INFO:teuthology.misc:Compressing logs... 2022-02-10T14:33:28.714 INFO:teuthology.kill:No teuthology processes running 2022-02-10T14:33:29.868 INFO:teuthology.kill:Nuking machines: ['smithi136', 'smithi173'] 2022-02-10T14:33:30.888 INFO:teuthology.kill:2022-02-10 14:33:30,888.888 INFO:teuthology.nuke:targets: 2022-02-10T14:33:30.889 INFO:teuthology.kill: smithi136.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/g5vTBjrKRTlyso3FA9WqWO4kx7gFm7p8lKTJxSJBbpCaILaj5zFw6MfmnpYK4MFBLV4rJfv447J4Ci+K3BmLqX2CtIWZLN+lQhNbbw4QCEVDIz8FBg0Ut1qeBiU02l4Q0O2l7GjKf5G/PCY4xiGI66vfE94mMXEtZwfVNb4mULogM8If9QE33D8j9usykphZFQKC4oK2/5f54Y3OGxLgGPh471tOy6TGQO22vqn6d6GawKJ+EJwrujyO2EEGLAkXj3j+jByM+XP3aD0HoHX7b/Szdb9qZTT6HK7e83GyVLPE2u1NsVkFycDmVMdVwnvPXTFjumK1ZM5bVtPmn0YdeJt0fHpYsEUR65Xy6jr75LWGbnmQmRWogYpVu4Zr9HIoQ4xDxHGEhhaH0Tdg0U8Xfhou26Fi8/Zt30/Uy5PvSsPSqDFrK6SR57iPOEwf5I0YJFF5+SffZOgigd66o6eiLLT7ABxVl9iQDiXFJsJnWSkVifH5yZB/41GI+AyG1OE= 2022-02-10T14:33:30.890 INFO:teuthology.kill: smithi173.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQw0wG+0fUOxs4p9H7QAP7RXirRiCTHfbpYEilJ1EGwXJpaXwXwmU+4BBwA37/yE1XmIBLtR2nhVcjlBas5adQB+Mzot67jVqkvMYekcM54IIg0TzX5hNgKMxs4SgadM6wTjQPSA2LDWOjSRb1eog27d8CSkybkB6wT3s9pE6wunpb/MzQRwYZzCdS+OxEevPZeC0ImRzlmZX2JoQqbm3nO+//aC+ozQnaH8pDBsiNRdZaBczuYgxq/A1J6T8i8xzBt98xR3tcKMKhDgaYg2FRVm1Nqh20wJozOHvN34E7kWzuDGADmD9Wc/LDMr258GYkY3klHW0Oz45OwNN3Laz29AkLCX9jwM+T3HcdM5fsMHu7R6w68JUEWgM3Nuy8oO0m1YVciv03Z1Egju3qV1Mj1HgBX3Hj1KqtKcRh1UYXw01TPiktNukq+UfTHgeF0MoK4pxpo36ItYKRR/QwzU/wciPNAXxEfLQvU+mqyyBBfEi1f+A6DVIKvWFH8VHZk/c= 2022-02-10T14:33:30.916 INFO:teuthology.kill:2022-02-10 14:33:30,916.916 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-10T14:33:30.920 INFO:teuthology.kill:2022-02-10 14:33:30,920.920 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-10T14:33:30.998 INFO:teuthology.kill:2022-02-10 14:33:30,998.998 INFO:teuthology.orchestra.console:Power off smithi136 2022-02-10T14:33:31.013 INFO:teuthology.kill:2022-02-10 14:33:31,013.013 INFO:teuthology.orchestra.console:Power off smithi173 2022-02-10T14:33:39.367 INFO:teuthology.kill:2022-02-10 14:33:39,366.366 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2022-02-10T14:33:39.513 INFO:teuthology.kill:2022-02-10 14:33:39,512.512 INFO:teuthology.lock.ops:unlocked: smithi136.front.sepia.ceph.com 2022-02-10T14:33:43.498 INFO:teuthology.kill:2022-02-10 14:33:43,497.497 INFO:teuthology.orchestra.console:Power off for smithi173 completed 2022-02-10T14:33:43.644 INFO:teuthology.kill:2022-02-10 14:33:43,644.644 INFO:teuthology.lock.ops:unlocked: smithi173.front.sepia.ceph.com 2022-02-10T14:35:43.959 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-02-10T14:35:44.080 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi136 but it was locked by another job: /home/teuthworker/archive/matan-2022-02-10_14:11:48-rados:thrash-master-distro-basic-smithi/6673703 2022-02-10T14:35:44.080 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi173 but it was locked by another job: /home/teuthworker/archive/matan-2022-02-10_14:11:48-rados:thrash-master-distro-basic-smithi/6673703