2021-12-05T20:09:49.834 INFO:root:teuthology version: 1.1.0-844d1377 2021-12-05T20:09:49.963 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi140.front.sepia.ceph.com' reimaging 2021-12-05T20:09:49.964 INFO:teuthology.lock.ops:Updating [ubuntu@smithi140.front.sepia.ceph.com]: reset os type and version on server 2021-12-05T20:09:49.964 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2021-12-05T20:09:49.984 INFO:teuthology.lock.ops:Node 'ubuntu@smithi140.front.sepia.ceph.com' reimaging is complete 2021-12-05T20:09:49.984 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi176.front.sepia.ceph.com' reimaging 2021-12-05T20:09:49.984 INFO:teuthology.lock.ops:Updating [ubuntu@smithi176.front.sepia.ceph.com]: reset os type and version on server 2021-12-05T20:09:49.985 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2021-12-05T20:09:50.004 INFO:teuthology.lock.ops:Node 'ubuntu@smithi176.front.sepia.ceph.com' reimaging is complete 2021-12-05T20:09:50.193 INFO:teuthology.provision.fog.smithi140:Scheduling deploy of centos 8.stream 2021-12-05T20:09:50.206 INFO:teuthology.provision.fog.smithi176:Scheduling deploy of centos 8.stream 2021-12-05T20:09:50.546 INFO:teuthology.orchestra.console:Power off smithi140 2021-12-05T20:09:50.562 INFO:teuthology.orchestra.console:Power off smithi176 2021-12-05T20:09:58.916 INFO:teuthology.orchestra.console:Power off for smithi140 completed 2021-12-05T20:09:58.921 INFO:teuthology.orchestra.console:Power off for smithi176 completed 2021-12-05T20:09:59.018 INFO:teuthology.orchestra.console:Power on smithi140 2021-12-05T20:09:59.027 INFO:teuthology.orchestra.console:Power on smithi176 2021-12-05T20:10:07.416 INFO:teuthology.orchestra.console:Power on for smithi176 completed 2021-12-05T20:10:07.456 INFO:teuthology.orchestra.console:Power on for smithi140 completed 2021-12-05T20:10:07.518 INFO:teuthology.provision.fog.smithi176:Waiting for deploy to finish 2021-12-05T20:10:07.557 INFO:teuthology.provision.fog.smithi140:Waiting for deploy to finish 2021-12-05T20:14:57.030 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-05T20:14:57.311 INFO:teuthology.provision.fog.smithi176:Node is ready 2021-12-05T20:14:57.329 INFO:teuthology.orchestra.run.smithi176.stdout:smithi176.front.sepia.ceph.com 2021-12-05T20:14:57.387 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176 smithi176.front.sepia.ceph.com smithi176 2021-12-05T20:14:57.724 INFO:teuthology.provision.fog.smithi176:Deploy complete! 2021-12-05T20:15:12.095 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-05T20:15:12.362 INFO:teuthology.provision.fog.smithi140:Node is ready 2021-12-05T20:15:12.379 INFO:teuthology.orchestra.run.smithi140.stdout:smithi140.front.sepia.ceph.com 2021-12-05T20:15:12.436 INFO:teuthology.orchestra.run.smithi140.stdout:172.21.15.140 smithi140.front.sepia.ceph.com smithi140 2021-12-05T20:15:12.784 INFO:teuthology.provision.fog.smithi140:Deploy complete! 2021-12-05T20:15:12.965 INFO:teuthology.lock.ops:Checking smithi140.front.sepia.ceph.com 2021-12-05T20:15:12.966 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-05T20:15:12.985 INFO:teuthology.lock.ops:Checking smithi176.front.sepia.ceph.com 2021-12-05T20:15:12.985 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-05T20:15:13.003 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: set os type and version on server 2021-12-05T20:15:13.004 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi140.front.sepia.ceph.com' 2021-12-05T20:15:13.190 INFO:teuthology.orchestra.run.smithi140.stdout:x86_64 2021-12-05T20:15:13.246 INFO:teuthology.orchestra.run.smithi140.stdout:NAME="CentOS Stream" 2021-12-05T20:15:13.246 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION="8" 2021-12-05T20:15:13.246 INFO:teuthology.orchestra.run.smithi140.stdout:ID="centos" 2021-12-05T20:15:13.247 INFO:teuthology.orchestra.run.smithi140.stdout:ID_LIKE="rhel fedora" 2021-12-05T20:15:13.247 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION_ID="8" 2021-12-05T20:15:13.247 INFO:teuthology.orchestra.run.smithi140.stdout:PLATFORM_ID="platform:el8" 2021-12-05T20:15:13.247 INFO:teuthology.orchestra.run.smithi140.stdout:PRETTY_NAME="CentOS Stream 8" 2021-12-05T20:15:13.247 INFO:teuthology.orchestra.run.smithi140.stdout:ANSI_COLOR="0;31" 2021-12-05T20:15:13.248 INFO:teuthology.orchestra.run.smithi140.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-05T20:15:13.248 INFO:teuthology.orchestra.run.smithi140.stdout:HOME_URL="https://centos.org/" 2021-12-05T20:15:13.248 INFO:teuthology.orchestra.run.smithi140.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-12-05T20:15:13.248 INFO:teuthology.orchestra.run.smithi140.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-12-05T20:15:13.249 INFO:teuthology.orchestra.run.smithi140.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-12-05T20:15:13.250 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2021-12-05T20:15:13.269 INFO:teuthology.lock.ops:Updating [smithi176.front.sepia.ceph.com]: set os type and version on server 2021-12-05T20:15:13.269 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2021-12-05T20:15:13.511 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2021-12-05T20:15:13.567 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="CentOS Stream" 2021-12-05T20:15:13.568 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="8" 2021-12-05T20:15:13.568 INFO:teuthology.orchestra.run.smithi176.stdout:ID="centos" 2021-12-05T20:15:13.568 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE="rhel fedora" 2021-12-05T20:15:13.568 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="8" 2021-12-05T20:15:13.569 INFO:teuthology.orchestra.run.smithi176.stdout:PLATFORM_ID="platform:el8" 2021-12-05T20:15:13.569 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="CentOS Stream 8" 2021-12-05T20:15:13.569 INFO:teuthology.orchestra.run.smithi176.stdout:ANSI_COLOR="0;31" 2021-12-05T20:15:13.569 INFO:teuthology.orchestra.run.smithi176.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-05T20:15:13.569 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://centos.org/" 2021-12-05T20:15:13.570 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-12-05T20:15:13.570 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-12-05T20:15:13.570 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-12-05T20:15:13.571 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2021-12-05T20:15:13.666 INFO:teuthology.dispatcher.supervisor:Running job 6546877 2021-12-05T20:15:13.667 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi/6546877 --name benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi --description rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{centos_8.stream} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} -- /home/teuthworker/archive/benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi/6546877/orig.config.yaml 2021-12-05T20:15:13.670 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-12-05_15:06:27-rados-WIP_GBH_safe_fast_shutdown_1-distro-basic-smithi/6546877 2021-12-05T20:15:13.671 INFO:teuthology.dispatcher.supervisor:Job PID: 9261 2021-12-05T20:15:13.672 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-12-06T08:15:33.734 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-12-06T08:15:33.926 INFO:teuthology.kill:Killing Pids: {9261} 2021-12-06T08:15:35.050 INFO:teuthology.task.internal:roles: ubuntu@smithi140.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-12-06T08:15:35.051 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-12-06T08:15:35.052 INFO:teuthology.misc:Compressing logs... 2021-12-06T08:15:35.052 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi140.front.sepia.ceph.com' 2021-12-06T08:15:35.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2021-12-06T08:15:35.545 INFO:teuthology.misc:Compressing logs... 2021-12-06T08:15:36.378 INFO:teuthology.orchestra.run.smithi176.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-12-06T08:15:36.603 INFO:teuthology.orchestra.run.smithi140.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-12-06T08:15:40.689 INFO:teuthology.misc:Compressing logs... 2021-12-06T08:16:15.778 INFO:teuthology.kill:No teuthology processes running 2021-12-06T08:16:16.885 INFO:teuthology.kill:Nuking machines: ['smithi140', 'smithi176'] 2021-12-06T08:16:17.845 INFO:teuthology.kill:2021-12-06 08:16:17,845.845 INFO:teuthology.nuke:targets: 2021-12-06T08:16:17.898 INFO:teuthology.kill: smithi140.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAfopxcqqie4yk0JqnlypXw8Z0W3af7BoAoZN8pCGTUKxbQBbUdcJ34QbFJ08LM2MK1f4bf7aF5stuphh/fzDhu4h+1XbbMiTV+Lwqp7r+GYaLjkQO9YYId02tmYKweKgBvHOoEKtixV9fCxqkZ8q2JmP6aKlImkVVrq355+KqUynUtz8zXIP8jdLogWyI2ZVM5eK2c7qjIAW6wA/vyuPhg95kCUBcJuoMbM4vD1FR+fwRi3C1MNCa7tYZg/WM2AbPjGpxmgfJPW4fQQ1o296ZUEHYy6+LftmFkyCNQeUzzO8FsYmW7hSt616c+2QITrdEveuZLipeBb2O3FLBpJvpmszKeKeqmFYrjFnn2mGMPg/zGwYA4hn57R5yt3BwPZTNDFa4NmQCtK1VHE+/5v7YIMGqfT8nu4ekNHXZBirVU2OjlrPs3O8mIUHkkcyGHeQcZe3VMdFZazuIZchc5JEz1wroFFZ92HIalR0EI4gbCgw/BjPGtA/FfbRra/I2QZs= 2021-12-06T08:16:17.899 INFO:teuthology.kill: smithi176.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDU62xFxlr7wWu35ODGvJ7NRbS1Xb29URAUe6GCQYZ1f1BkdHDqFU3pc1lN6UhcJMLM22j4JYfmO//+sKpvlH302fJGvMRYzsAmTFunbSThjBre/f6/T3X+te1fvHwjRJnZKyOgMiNC/WmnSy7/l2Mbfpv/H5jkUSOc/tVQmsUCDDAXpT14lDCTn/TEY7JCpo/s9qd00hIoxozxG0wys8AdWQY6rsDzkn9L3qXUCMAfcWCPaP3x8Mb/25TrmF77irDn5z0QCdMFyTkco2LxTn+HX6D2zZdXmT0IEs7aVI7FPn0vV1Wy7Or+15yJE7qU4pZS9o2ejKdWQVSbvoRYa4mhq/9qxKnbQdC6LWJYfr5UKfDxUwSBqm5IIYFuuNpUVjkLY99Y1s0ei4MjTVDZbXjTqeHa6w+s8Tbm3hAgyEFdMDQJlibHZKD3WUGJKX+mK8Y/8hZ5h6P46cI9HYJ0PzgW7pByJCzlxFlPHamphC/BA2dsqWfSw/ag+k7nvOuPLJE= 2021-12-06T08:16:17.899 INFO:teuthology.kill:2021-12-06 08:16:17,872.872 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-06T08:16:17.899 INFO:teuthology.kill:2021-12-06 08:16:17,875.875 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-06T08:16:17.953 INFO:teuthology.kill:2021-12-06 08:16:17,953.953 INFO:teuthology.orchestra.console:Power off smithi140 2021-12-06T08:16:17.966 INFO:teuthology.kill:2021-12-06 08:16:17,966.966 INFO:teuthology.orchestra.console:Power off smithi176 2021-12-06T08:16:26.331 INFO:teuthology.kill:2021-12-06 08:16:26,330.330 INFO:teuthology.orchestra.console:Power off for smithi176 completed 2021-12-06T08:16:26.477 INFO:teuthology.kill:2021-12-06 08:16:26,476.476 INFO:teuthology.lock.ops:unlocked: smithi176.front.sepia.ceph.com 2021-12-06T08:16:30.437 INFO:teuthology.kill:2021-12-06 08:16:30,437.437 INFO:teuthology.orchestra.console:Power off for smithi140 completed 2021-12-06T08:16:30.581 INFO:teuthology.kill:2021-12-06 08:16:30,581.581 INFO:teuthology.lock.ops:unlocked: smithi140.front.sepia.ceph.com 2021-12-06T08:18:30.958 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-12-06T08:18:31.072 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi140 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-06_02:01:02-rbd-master-distro-default-smithi/6547224 2021-12-06T08:18:31.073 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi176 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-06_02:01:02-rbd-master-distro-default-smithi/6547224