2021-12-06T17:04:57.303 INFO:root:teuthology version: 1.1.0-844d1377 2021-12-06T17:04:57.420 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi008.front.sepia.ceph.com' reimaging 2021-12-06T17:04:57.421 INFO:teuthology.lock.ops:Updating [ubuntu@smithi008.front.sepia.ceph.com]: reset os type and version on server 2021-12-06T17:04:57.422 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2021-12-06T17:04:57.439 INFO:teuthology.lock.ops:Node 'ubuntu@smithi008.front.sepia.ceph.com' reimaging is complete 2021-12-06T17:04:57.440 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi037.front.sepia.ceph.com' reimaging 2021-12-06T17:04:57.440 INFO:teuthology.lock.ops:Updating [ubuntu@smithi037.front.sepia.ceph.com]: reset os type and version on server 2021-12-06T17:04:57.440 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2021-12-06T17:04:57.458 INFO:teuthology.lock.ops:Node 'ubuntu@smithi037.front.sepia.ceph.com' reimaging is complete 2021-12-06T17:04:57.655 INFO:teuthology.provision.fog.smithi008:Scheduling deploy of centos 8.stream 2021-12-06T17:04:57.679 INFO:teuthology.provision.fog.smithi037:Scheduling deploy of centos 8.stream 2021-12-06T17:04:57.989 INFO:teuthology.orchestra.console:Power off smithi008 2021-12-06T17:04:58.047 INFO:teuthology.orchestra.console:Power off smithi037 2021-12-06T17:05:06.359 INFO:teuthology.orchestra.console:Power off for smithi008 completed 2021-12-06T17:05:06.407 INFO:teuthology.orchestra.console:Power off for smithi037 completed 2021-12-06T17:05:06.461 INFO:teuthology.orchestra.console:Power on smithi008 2021-12-06T17:05:06.508 INFO:teuthology.orchestra.console:Power on smithi037 2021-12-06T17:05:14.830 INFO:teuthology.orchestra.console:Power on for smithi008 completed 2021-12-06T17:05:14.931 INFO:teuthology.provision.fog.smithi008:Waiting for deploy to finish 2021-12-06T17:05:14.943 INFO:teuthology.orchestra.console:Power on for smithi037 completed 2021-12-06T17:05:15.045 INFO:teuthology.provision.fog.smithi037:Waiting for deploy to finish 2021-12-06T17:09:49.560 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-06T17:09:49.851 INFO:teuthology.provision.fog.smithi037:Node is ready 2021-12-06T17:09:49.869 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037.front.sepia.ceph.com 2021-12-06T17:09:49.928 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37 smithi037.front.sepia.ceph.com smithi037 2021-12-06T17:09:50.262 INFO:teuthology.provision.fog.smithi037:Deploy complete! 2021-12-06T17:10:04.583 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-06T17:10:04.874 INFO:teuthology.provision.fog.smithi008:Node is ready 2021-12-06T17:10:04.892 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008.front.sepia.ceph.com 2021-12-06T17:10:04.949 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8 smithi008.front.sepia.ceph.com smithi008 2021-12-06T17:10:05.288 INFO:teuthology.provision.fog.smithi008:Deploy complete! 2021-12-06T17:10:05.439 INFO:teuthology.lock.ops:Checking smithi008.front.sepia.ceph.com 2021-12-06T17:10:05.440 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-06T17:10:05.457 INFO:teuthology.lock.ops:Checking smithi037.front.sepia.ceph.com 2021-12-06T17:10:05.458 INFO:teuthology.lock.ops:New key found. Updating... 2021-12-06T17:10:05.479 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: set os type and version on server 2021-12-06T17:10:05.479 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2021-12-06T17:10:05.666 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64 2021-12-06T17:10:05.723 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="CentOS Stream" 2021-12-06T17:10:05.724 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="8" 2021-12-06T17:10:05.724 INFO:teuthology.orchestra.run.smithi008.stdout:ID="centos" 2021-12-06T17:10:05.724 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE="rhel fedora" 2021-12-06T17:10:05.724 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="8" 2021-12-06T17:10:05.725 INFO:teuthology.orchestra.run.smithi008.stdout:PLATFORM_ID="platform:el8" 2021-12-06T17:10:05.725 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="CentOS Stream 8" 2021-12-06T17:10:05.725 INFO:teuthology.orchestra.run.smithi008.stdout:ANSI_COLOR="0;31" 2021-12-06T17:10:05.725 INFO:teuthology.orchestra.run.smithi008.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-06T17:10:05.726 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://centos.org/" 2021-12-06T17:10:05.726 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-12-06T17:10:05.726 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-12-06T17:10:05.726 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-12-06T17:10:05.728 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2021-12-06T17:10:05.746 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: set os type and version on server 2021-12-06T17:10:05.747 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2021-12-06T17:10:05.998 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2021-12-06T17:10:06.055 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2021-12-06T17:10:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="8" 2021-12-06T17:10:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2021-12-06T17:10:06.056 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2021-12-06T17:10:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="8" 2021-12-06T17:10:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el8" 2021-12-06T17:10:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 8" 2021-12-06T17:10:06.057 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2021-12-06T17:10:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-12-06T17:10:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2021-12-06T17:10:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-12-06T17:10:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2021-12-06T17:10:06.058 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2021-12-06T17:10:06.059 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2021-12-06T17:10:06.150 INFO:teuthology.dispatcher.supervisor:Running job 6547759 2021-12-06T17:10:06.150 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547759 --name benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi --description rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} -- /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547759/orig.config.yaml 2021-12-06T17:10:06.153 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-12-06_16:41:40-rados-WIP_GBH_safe_fast_shutdown_2-distro-basic-smithi/6547759 2021-12-06T17:10:06.154 INFO:teuthology.dispatcher.supervisor:Job PID: 15317 2021-12-06T17:10:06.154 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-12-07T05:10:26.803 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-12-07T05:10:26.986 INFO:teuthology.kill:Killing Pids: {15317} 2021-12-07T05:10:28.171 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2021-12-07T05:10:28.172 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2021-12-07T05:10:28.172 INFO:teuthology.misc:Compressing logs... 2021-12-07T05:10:28.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2021-12-07T05:10:28.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2021-12-07T05:10:28.682 INFO:teuthology.misc:Compressing logs... 2021-12-07T05:10:28.939 INFO:teuthology.orchestra.run.smithi037.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-12-07T05:10:32.163 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /home/ubuntu/cephtest/archive/syslog/misc.log: file size changed while zipping 2021-12-07T05:10:33.159 INFO:teuthology.misc:Compressing logs... 2021-12-07T05:10:38.618 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/2fd851c2-56b8-11ec-8c2e-001a4aab830c/ceph-mon.a.log: file size changed while zipping 2021-12-07T05:10:40.132 INFO:teuthology.orchestra.run.smithi037.stderr:gzip: /var/log/ceph/2fd851c2-56b8-11ec-8c2e-001a4aab830c/ceph-osd.7.log: file size changed while zipping 2021-12-07T05:10:45.890 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/2fd851c2-56b8-11ec-8c2e-001a4aab830c/ceph-osd.0.log: file size changed while zipping 2021-12-07T05:10:47.194 INFO:teuthology.orchestra.run.smithi037.stderr:gzip: /var/log/ceph/2fd851c2-56b8-11ec-8c2e-001a4aab830c/ceph-osd.5.log: file size changed while zipping 2021-12-07T05:10:49.565 INFO:teuthology.orchestra.run.smithi037.stderr:gzip: /var/log/ceph/2fd851c2-56b8-11ec-8c2e-001a4aab830c/ceph-mon.b.log: file size changed while zipping 2021-12-07T05:10:51.530 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/2fd851c2-56b8-11ec-8c2e-001a4aab830c/ceph-osd.2.log: file size changed while zipping 2021-12-07T05:10:56.144 INFO:teuthology.orchestra.run.smithi037.stderr:gzip: /var/log/ceph/2fd851c2-56b8-11ec-8c2e-001a4aab830c/ceph-osd.4.log: file size changed while zipping 2021-12-07T05:10:59.616 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/2fd851c2-56b8-11ec-8c2e-001a4aab830c/ceph-osd.1.log: file size changed while zipping 2021-12-07T05:11:01.367 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/2fd851c2-56b8-11ec-8c2e-001a4aab830c/ceph-mgr.y.log: file size changed while zipping 2021-12-07T05:11:01.656 INFO:teuthology.orchestra.run.smithi037.stderr:gzip: /var/log/ceph/2fd851c2-56b8-11ec-8c2e-001a4aab830c/ceph-osd.6.log: file size changed while zipping 2021-12-07T05:11:03.575 INFO:teuthology.orchestra.run.smithi008.stderr:gzip: /var/log/ceph/2fd851c2-56b8-11ec-8c2e-001a4aab830c/ceph-mon.c.log: file size changed while zipping 2021-12-07T05:12:32.923 INFO:teuthology.kill:No teuthology processes running 2021-12-07T05:12:34.033 INFO:teuthology.kill:Nuking machines: ['smithi008', 'smithi037'] 2021-12-07T05:12:34.956 INFO:teuthology.kill:2021-12-07 05:12:34,956.956 INFO:teuthology.nuke:targets: 2021-12-07T05:12:34.957 INFO:teuthology.kill: smithi008.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCisPJmHsFSpXLyGZ8H3dD4sHGMVB1UkG6PWneP9WEoXQc1wgl1cqkQFY9cElk0EHV6I6uH/5hs5nKjl3454mqHXS64ts2Bb9iVZFwQW/XBhoTEmEzqB9uN5lYa7jEpHW7aVmACodrJSGifYkA1cM6mE1Fe592hXMugXflMRbfRLiTdubBTRi7djIJyDxvredegwBBBJpOo/PyBVgpdKQ+gUeKN+ZDbBRPuJrSGju5mtUIj0tCC8cxmuV1qs8c0ZBXcO4eVyZOAewDr/rY/jPc4qh7dNlcHS2FUBZRJydnisThZzf71/D/l1CeLb3Q67b5mi6QA0DIel+xzHFihndiBhK2gt+AgRoLFX0DzxCjbPeIwSun5/Y5gWGGEq8M7WO6xzbK+gjhaUIoSwGFjKgmiPudCfI749+DeW2gtVy5Y1cWEzMM7ZrtkuVr26OaIepQKrtPQSkkwkmevqwJtJ9V4q50OO0hxSpQaWDwSrHsBWit1zRBi64GhU/+Fg9T1Qv8= 2021-12-07T05:12:34.957 INFO:teuthology.kill: smithi037.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9JeRRbMB9aexntVx3LZuFoUc4Mz43Xja9aTUWcEfsXH9FY/KwpzlqVaCwfSZwJyCA3BLgZRhfV/uRpnsoRQoExi5eJBfhLneKVqrtKaoxdtCwMjHjdQQEkudIvj2hFFGIVJgAjcqfcKYNz3OYwfDox+Vb/f3mH2qyj68Qcv0a7z1yHfhAAyATIwr6OTLDB7gCA7mt3q5G9r/y+VWVuvNU+MSEkY2LlE3SNlw4m45ccgUXoC+NK7cSVu23gW+UgKhKYnTLHmfI8TtGmqS7VDxTR46nqNXFPDOx0FCyyKZLq6q8kgV69h1x6KIitb8REEIGeicjbFJnhEDapULmavrXnN9doFhod74ZzFSkKrZEybBpBYgjiTQ41x2DgqTNk9O1SadNlhEWSm9qmfZtIjxbknFyIL7uv+l2RoNIIa8rVmVQjr3dhJuh/WMdBQ2vxq6WTUbV0nzsSDxkH5d3RijPoijSgzwwkTsUCGnLtYrASI4ZnQZjHyoUN1kBWStX0ss= 2021-12-07T05:12:34.982 INFO:teuthology.kill:2021-12-07 05:12:34,982.982 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-07T05:12:34.986 INFO:teuthology.kill:2021-12-07 05:12:34,986.986 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-07T05:12:35.069 INFO:teuthology.kill:2021-12-07 05:12:35,068.068 INFO:teuthology.orchestra.console:Power off smithi008 2021-12-07T05:12:35.084 INFO:teuthology.kill:2021-12-07 05:12:35,083.083 INFO:teuthology.orchestra.console:Power off smithi037 2021-12-07T05:12:43.452 INFO:teuthology.kill:2021-12-07 05:12:43,451.451 INFO:teuthology.orchestra.console:Power off for smithi008 completed 2021-12-07T05:12:43.457 INFO:teuthology.kill:2021-12-07 05:12:43,457.457 INFO:teuthology.orchestra.console:Power off for smithi037 completed 2021-12-07T05:12:43.603 INFO:teuthology.kill:2021-12-07 05:12:43,602.602 INFO:teuthology.lock.ops:unlocked: smithi037.front.sepia.ceph.com 2021-12-07T05:12:43.606 INFO:teuthology.kill:2021-12-07 05:12:43,606.606 INFO:teuthology.lock.ops:unlocked: smithi008.front.sepia.ceph.com 2021-12-07T05:14:43.898 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-12-07T05:14:43.994 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi008 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-07_05:00:02-smoke-master-distro-default-smithi/6549835 2021-12-07T05:14:43.995 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi037 but it was locked by another job: /home/teuthworker/archive/teuthology-2021-12-07_05:00:02-smoke-master-distro-default-smithi/6549835