2022-03-13T04:44:52.687 INFO:root:teuthology version: 1.1.0-fbfc7b38 2022-03-13T04:44:52.828 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi027.front.sepia.ceph.com' reimaging 2022-03-13T04:44:52.830 INFO:teuthology.lock.ops:Updating [ubuntu@smithi027.front.sepia.ceph.com]: reset os type and version on server 2022-03-13T04:44:52.830 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2022-03-13T04:44:52.849 INFO:teuthology.lock.ops:Node 'ubuntu@smithi027.front.sepia.ceph.com' reimaging is complete 2022-03-13T04:44:52.849 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi077.front.sepia.ceph.com' reimaging 2022-03-13T04:44:52.850 INFO:teuthology.lock.ops:Updating [ubuntu@smithi077.front.sepia.ceph.com]: reset os type and version on server 2022-03-13T04:44:52.850 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2022-03-13T04:44:52.869 INFO:teuthology.lock.ops:Node 'ubuntu@smithi077.front.sepia.ceph.com' reimaging is complete 2022-03-13T04:44:53.081 INFO:teuthology.provision.fog.smithi077:Scheduling deploy of centos 8.stream 2022-03-13T04:44:53.083 INFO:teuthology.provision.fog.smithi027:Scheduling deploy of centos 8.stream 2022-03-13T04:44:53.392 INFO:teuthology.orchestra.console:Power off smithi027 2022-03-13T04:44:53.432 INFO:teuthology.orchestra.console:Power off smithi077 2022-03-13T04:45:01.771 INFO:teuthology.orchestra.console:Power off for smithi027 completed 2022-03-13T04:45:01.872 INFO:teuthology.orchestra.console:Power on smithi027 2022-03-13T04:45:05.927 INFO:teuthology.orchestra.console:Power off for smithi077 completed 2022-03-13T04:45:06.028 INFO:teuthology.orchestra.console:Power on smithi077 2022-03-13T04:45:10.254 INFO:teuthology.orchestra.console:Power on for smithi027 completed 2022-03-13T04:45:10.284 INFO:teuthology.orchestra.console:Power on for smithi077 completed 2022-03-13T04:45:10.356 INFO:teuthology.provision.fog.smithi027:Waiting for deploy to finish 2022-03-13T04:45:10.385 INFO:teuthology.provision.fog.smithi077:Waiting for deploy to finish 2022-03-13T04:47:41.403 ERROR:teuthology.orchestra.connection:Error authenticating with smithi077.front.sepia.ceph.com: Authentication failed. 2022-03-13T04:49:22.108 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-13T04:49:47.306 INFO:teuthology.provision.fog.smithi077:Node is ready 2022-03-13T04:49:47.329 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077.front.sepia.ceph.com 2022-03-13T04:49:47.388 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77 smithi077.front.sepia.ceph.com smithi077 2022-03-13T04:49:47.760 INFO:teuthology.provision.fog.smithi077:Deploy complete! 2022-03-13T04:50:46.938 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-03-13T04:51:07.127 INFO:teuthology.provision.fog.smithi027:Node is ready 2022-03-13T04:51:07.149 INFO:teuthology.orchestra.run.smithi027.stdout:smithi107 2022-03-13T04:51:07.209 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.107 smithi107.front.sepia.ceph.com smithi107 2022-03-13T04:51:07.614 INFO:teuthology.provision.fog.smithi027:Deploy complete! 2022-03-13T04:51:07.810 INFO:teuthology.lock.ops:Checking smithi027.front.sepia.ceph.com 2022-03-13T04:51:07.811 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-13T04:51:07.828 INFO:teuthology.lock.ops:Checking smithi077.front.sepia.ceph.com 2022-03-13T04:51:07.828 INFO:teuthology.lock.ops:New key found. Updating... 2022-03-13T04:51:07.845 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: set os type and version on server 2022-03-13T04:51:07.846 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2022-03-13T04:51:08.054 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2022-03-13T04:51:08.110 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2022-03-13T04:51:08.111 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="8" 2022-03-13T04:51:08.111 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2022-03-13T04:51:08.111 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2022-03-13T04:51:08.111 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="8" 2022-03-13T04:51:08.112 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el8" 2022-03-13T04:51:08.112 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-13T04:51:08.112 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2022-03-13T04:51:08.112 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-13T04:51:08.113 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2022-03-13T04:51:08.113 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-13T04:51:08.113 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-13T04:51:08.113 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-13T04:51:08.115 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2022-03-13T04:51:08.132 INFO:teuthology.lock.ops:Updating [smithi077.front.sepia.ceph.com]: set os type and version on server 2022-03-13T04:51:08.132 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2022-03-13T04:51:08.335 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2022-03-13T04:51:08.395 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2022-03-13T04:51:08.395 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="8" 2022-03-13T04:51:08.396 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2022-03-13T04:51:08.396 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2022-03-13T04:51:08.396 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="8" 2022-03-13T04:51:08.396 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el8" 2022-03-13T04:51:08.397 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 8" 2022-03-13T04:51:08.397 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2022-03-13T04:51:08.397 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-03-13T04:51:08.397 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2022-03-13T04:51:08.398 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-03-13T04:51:08.398 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-03-13T04:51:08.398 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-03-13T04:51:08.399 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2022-03-13T04:51:08.492 INFO:teuthology.dispatcher.supervisor:Running job 6734089 2022-03-13T04:51:08.492 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_fbfc7b381c1d662565abb9d3552d468766bf123c/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-03-13_03:31:03-rados-pacific-distro-default-smithi/6734089 --name teuthology-2022-03-13_03:31:03-rados-pacific-distro-default-smithi --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/lockdep} -- /home/teuthworker/archive/teuthology-2022-03-13_03:31:03-rados-pacific-distro-default-smithi/6734089/orig.config.yaml 2022-03-13T04:51:08.496 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-03-13_03:31:03-rados-pacific-distro-default-smithi/6734089 2022-03-13T04:51:08.496 INFO:teuthology.dispatcher.supervisor:Job PID: 9443 2022-03-13T04:51:08.497 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-03-13T11:21:19.760 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-03-13T11:21:19.923 INFO:teuthology.kill:Killing Pids: {9443} 2022-03-13T11:21:21.132 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2022-03-13T11:21:21.133 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2022-03-13T11:21:21.133 INFO:teuthology.misc:Compressing logs... 2022-03-13T11:21:21.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2022-03-13T11:21:21.331 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2022-03-13T11:21:21.627 INFO:teuthology.misc:Compressing logs... 2022-03-13T11:21:21.785 INFO:teuthology.misc:Compressing logs... 2022-03-13T11:21:50.458 INFO:teuthology.kill:No teuthology processes running 2022-03-13T11:21:51.605 INFO:teuthology.kill:Nuking machines: ['smithi027', 'smithi077'] 2022-03-13T11:21:52.655 INFO:teuthology.kill:2022-03-13 11:21:52,654.654 INFO:teuthology.nuke:targets: 2022-03-13T11:21:52.656 INFO:teuthology.kill: smithi027.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5dVnqiuD0IEntuO8+MEOn/puHGF4HWUIpMKHD+oD0G/dzn6BiqU8XlZWXMxiax6iH3ZJ09KgXv60SsAurYAAWXBpjrTUmDMc/RPc4yuYjreEKd2kzLe0wmn8gfYKrdGrxNk7sdlqWioJ/HnrEn3N3fSaA+JjgHRjRr71B4x9DZjXrZU/EXqafbGZEGwca5tkMytOS03MqQNKJHC2UcNxOV1Ze3rkiZmwYJ1W3PzocAXZk87bLMPqpfySNMWpMVLxaD0A+IE1y3PGLaxWOCjNeoLTiIBT9leeghL7ujJIY+T+exhjhfPhzT8Jo00yeDF03dGMn4vzEc3lsPgDB3cWif0fs5JwvvtpXyx2lFYr2DPwGHltMPsy1FIwwoDZoKKxGVmVQnhSy2Z4H63i9FTgyC/wk7bbLF0xO8yX1GuEBlcRj/ElTzcnFKU+DwG69CnczIyC2mPcueqDz/xdpTzKAN/dLU6/Qx4KbMf/FilkCwsGS2kfL7L7Wfv9GuJnXR3U= 2022-03-13T11:21:52.656 INFO:teuthology.kill: smithi077.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj5msXfYcme13PuOmpG/RoUaZMdCbdfcti13dUWnC/CerQtQSAxHKPT1D4Aq2ArGmo0oNHcPrIMktXXiP1BqhfUPKWob/lz3EDB1YUMl1EWRMsQ1vrHq7fxus48WtS74UDuJttwucQ18qNX7KxG9S2s9DMZ5BXEH7XqMF7paTWhzLijXG3q2hwRrYAjYC3+HBPvqv5o+GrutB4FOJaGhmANIUWUNzz4tV1n0Yo/TKzcSmRGVm5H938+AOu96nFM9wc7koJzQrS31RATuOujIJszIqOLI/tKkwNqlt+vkXwNRZV0WMl0oERyZgzF+Jzk7zSTqSbf2BzP3Bb6PV/3zMl2UO8hBVKLJwmQKt2Q+hwdzCLCTAlQh0H3HdRlCOtvbg6nWfyqOnp5w5kuWubcZe6KKG8B5U9mgK6Da1uCGxfQ+zncEPLT5wjSVootgwHMorNF+bmhEjd3Kg42QGXfohIdZfTejspzO/4hk8/5gIbyWp8fEfWbSC+CljSjNbtZlc= 2022-03-13T11:21:52.681 INFO:teuthology.kill:2022-03-13 11:21:52,680.680 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-13T11:21:52.684 INFO:teuthology.kill:2022-03-13 11:21:52,683.683 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-03-13T11:21:52.760 INFO:teuthology.kill:2022-03-13 11:21:52,760.760 INFO:teuthology.orchestra.console:Power off smithi027 2022-03-13T11:21:52.774 INFO:teuthology.kill:2022-03-13 11:21:52,773.773 INFO:teuthology.orchestra.console:Power off smithi077 2022-03-13T11:22:01.135 INFO:teuthology.kill:2022-03-13 11:22:01,135.135 INFO:teuthology.orchestra.console:Power off for smithi077 completed 2022-03-13T11:22:01.284 INFO:teuthology.kill:2022-03-13 11:22:01,283.283 INFO:teuthology.lock.ops:unlocked: smithi077.front.sepia.ceph.com 2022-03-13T11:22:05.240 INFO:teuthology.kill:2022-03-13 11:22:05,239.239 INFO:teuthology.orchestra.console:Power off for smithi027 completed 2022-03-13T11:22:05.395 INFO:teuthology.kill:2022-03-13 11:22:05,394.394 INFO:teuthology.lock.ops:unlocked: smithi027.front.sepia.ceph.com 2022-03-13T11:24:05.601 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-03-13T11:24:05.701 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi027 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-03-13_07:01:03-rados-master-distro-default-smithi/6734783 2022-03-13T11:24:05.701 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi077 but it was locked by another job: /home/teuthworker/archive/teuthology-2022-03-13_07:01:03-rados-master-distro-default-smithi/6734783