2021-08-16T18:41:05.227 INFO:root:teuthology version: 1.1.0-321319b1 2021-08-16T18:41:05.329 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi002.front.sepia.ceph.com' reimaging 2021-08-16T18:41:05.331 INFO:teuthology.lock.ops:Updating [ubuntu@smithi002.front.sepia.ceph.com]: reset os type and version on server 2021-08-16T18:41:05.331 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2021-08-16T18:41:05.348 INFO:teuthology.lock.ops:Node 'ubuntu@smithi002.front.sepia.ceph.com' reimaging is complete 2021-08-16T18:41:05.349 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi046.front.sepia.ceph.com' reimaging 2021-08-16T18:41:05.349 INFO:teuthology.lock.ops:Updating [ubuntu@smithi046.front.sepia.ceph.com]: reset os type and version on server 2021-08-16T18:41:05.349 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2021-08-16T18:41:05.367 INFO:teuthology.lock.ops:Node 'ubuntu@smithi046.front.sepia.ceph.com' reimaging is complete 2021-08-16T18:41:05.553 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of rhel 8.3 2021-08-16T18:41:05.562 INFO:teuthology.provision.fog.smithi046:Scheduling deploy of rhel 8.3 2021-08-16T18:41:05.886 INFO:teuthology.orchestra.console:Power off smithi002 2021-08-16T18:41:05.904 INFO:teuthology.orchestra.console:Power off smithi046 2021-08-16T18:41:14.264 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2021-08-16T18:41:14.268 INFO:teuthology.orchestra.console:Power off for smithi046 completed 2021-08-16T18:41:14.365 INFO:teuthology.orchestra.console:Power on smithi002 2021-08-16T18:41:14.374 INFO:teuthology.orchestra.console:Power on smithi046 2021-08-16T18:41:22.734 INFO:teuthology.orchestra.console:Power on for smithi002 completed 2021-08-16T18:41:22.738 INFO:teuthology.orchestra.console:Power on for smithi046 completed 2021-08-16T18:41:22.836 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish 2021-08-16T18:41:22.839 INFO:teuthology.provision.fog.smithi046:Waiting for deploy to finish 2021-08-16T18:45:30.552 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-08-16T18:45:30.868 INFO:teuthology.provision.fog.smithi002:Node is ready 2021-08-16T18:45:30.883 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com 2021-08-16T18:45:30.938 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002 2021-08-16T18:45:31.278 INFO:teuthology.provision.fog.smithi002:Deploy complete! 2021-08-16T18:45:54.556 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-08-16T18:45:54.883 INFO:teuthology.provision.fog.smithi046:Node is ready 2021-08-16T18:45:54.897 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046.front.sepia.ceph.com 2021-08-16T18:45:54.952 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2021-08-16T18:45:55.289 INFO:teuthology.provision.fog.smithi046:Deploy complete! 2021-08-16T18:45:55.456 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com 2021-08-16T18:45:55.458 INFO:teuthology.lock.ops:New key found. Updating... 2021-08-16T18:45:55.475 INFO:teuthology.lock.ops:Checking smithi046.front.sepia.ceph.com 2021-08-16T18:45:55.475 INFO:teuthology.lock.ops:New key found. Updating... 2021-08-16T18:45:55.490 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server 2021-08-16T18:45:55.491 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-16T18:45:55.785 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2021-08-16T18:45:55.839 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="Red Hat Enterprise Linux" 2021-08-16T18:45:55.840 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8.3 (Ootpa)" 2021-08-16T18:45:55.840 INFO:teuthology.orchestra.run.smithi002.stdout:ID="rhel" 2021-08-16T18:45:55.840 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="fedora" 2021-08-16T18:45:55.841 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8.3" 2021-08-16T18:45:55.841 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8" 2021-08-16T18:45:55.841 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-08-16T18:45:55.841 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2021-08-16T18:45:55.842 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-08-16T18:45:55.842 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://www.redhat.com/" 2021-08-16T18:45:55.842 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-08-16T18:45:55.842 INFO:teuthology.orchestra.run.smithi002.stdout: 2021-08-16T18:45:55.842 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-08-16T18:45:55.843 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-08-16T18:45:55.843 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-08-16T18:45:55.843 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-08-16T18:45:55.844 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2021-08-16T18:45:55.864 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: set os type and version on server 2021-08-16T18:45:55.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-16T18:45:56.154 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2021-08-16T18:45:56.209 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="Red Hat Enterprise Linux" 2021-08-16T18:45:56.209 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="8.3 (Ootpa)" 2021-08-16T18:45:56.209 INFO:teuthology.orchestra.run.smithi046.stdout:ID="rhel" 2021-08-16T18:45:56.209 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="fedora" 2021-08-16T18:45:56.210 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="8.3" 2021-08-16T18:45:56.210 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el8" 2021-08-16T18:45:56.210 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-08-16T18:45:56.210 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2021-08-16T18:45:56.211 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-08-16T18:45:56.211 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://www.redhat.com/" 2021-08-16T18:45:56.211 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-08-16T18:45:56.211 INFO:teuthology.orchestra.run.smithi046.stdout: 2021-08-16T18:45:56.211 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-08-16T18:45:56.212 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-08-16T18:45:56.212 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-08-16T18:45:56.212 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-08-16T18:45:56.213 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2021-08-16T18:45:56.285 INFO:teuthology.dispatcher.supervisor:Running job 6340974 2021-08-16T18:45:56.286 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/bin/teuthology -v --owner scheduled_rzarzynski@teuthology --archive /home/teuthworker/archive/rzarzynski-2021-08-16_18:34:22-rados-master-distro-basic-smithi/6340974 --name rzarzynski-2021-08-16_18:34:22-rados-master-distro-basic-smithi --description rados/cephadm/with-work/{0-distro/rhel_8.3_kubic_stable fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} -- /home/teuthworker/archive/rzarzynski-2021-08-16_18:34:22-rados-master-distro-basic-smithi/6340974/orig.config.yaml 2021-08-16T18:45:56.289 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rzarzynski-2021-08-16_18:34:22-rados-master-distro-basic-smithi/6340974 2021-08-16T18:45:56.290 INFO:teuthology.dispatcher.supervisor:Job PID: 18999 2021-08-16T18:45:56.290 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-08-17T06:46:48.063 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-08-17T06:46:48.268 INFO:teuthology.kill:Killing Pids: {18999} 2021-08-17T06:46:50.033 INFO:teuthology.kill:Nuking machines: ['smithi002', 'smithi046'] 2021-08-17T06:46:51.461 INFO:teuthology.kill:2021-08-17 06:46:51,460.460 INFO:teuthology.nuke:targets: 2021-08-17T06:46:51.462 INFO:teuthology.kill: smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBYFQCkfKXjojid/LA7aLMIhox/URnv+vOFL///TOBpMfMXnTrj+MR03ReLG8vh3tO9Jirm1mzpyUoVqamClaex6laov4CQNWS1+O2Qdt9qTOW8+zd0CIHaW94dTsqbfPzh6cQjT7U//kdj55HTQOXDTn8MCdmqKXoLY5q1gqXvMsf73a/yPZFDPsoh1XbQL8TXx0lXU4pObXQETknhJXVZWKmEUZPIM9OFCKlqWC+pNB/mzu9MaVBNoz7wcY6tyZhfQTcb8Mnd2Lf65Ul6ug+1SdJaXFJC3r4qJk1qGh0hP6g/nx/63hySt1KdD0m1cTD5qCMrtRqPC63B9RXnx5BviZ1iE9N0tfq2j9qaAADja2VkV8ClvWKxD+1OtLPK1MKGLpez4qAkGTr2uW55rAcWTvccrBoecauBejqr9Wf7ilGlrip5qeHtbMHVfOXRbtNiQExaR0U7zdSeizZQQVK9oOUylW3MGUT5xUcW1MXkNBK4IeFdAld7KD3iD13DLE= 2021-08-17T06:46:51.463 INFO:teuthology.kill: smithi046.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDU6wUZy/hc5n4i/RZ1RSbIHmipdIQPbpB8K7HfvvD3ChyYszcr2m1a8K2zU5wJ1SSFutR3g/5P6Ra0Bu13oObD6j6ZpP0zeD+p3gWf8x5L+rlgUyDiy20Vre3SxczHVGhtgDyIJBWgMfRLoZv7Rz6Hvy1VqbjvJnYSo4i+Tciw22uFIR9Gz7yaekM1lRcHKHvNwZlsWtUAysamS8QcURpMmw6qUAqQL4IXrbOhPDP5nJPQKfJ0Uf+LVnQkACELKJQZq/K0TUScE3QPOviznJ3uOGwp0hNsXN6kEM681n4RH/O+rCouE1K0ddRlbZZtW6cOdFwVTZ77kjSyNQXOBNACFsj7A0dMeXCOEAfPa8Hi+c8SqQlhNoP2wg0O7RPFddBtJuipG6vCgf/NnuOCzP3bbZ8zRIhXLUgyb0fP1/RRq7WbGwqc/Ij8mvr0Sq8LwWn/qLq1Om9khq2v4fd/Gv9QKY0G98foD5pbJQM9QdaS7bq7CgN2v6h7RRqSVTefKss= 2021-08-17T06:46:51.463 INFO:teuthology.kill:2021-08-17 06:46:51,461.461 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-17T06:46:51.467 INFO:teuthology.kill:2021-08-17 06:46:51,466.466 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-17T06:46:51.539 INFO:teuthology.kill:2021-08-17 06:46:51,539.539 INFO:teuthology.orchestra.console:Power off smithi046 2021-08-17T06:46:51.557 INFO:teuthology.kill:2021-08-17 06:46:51,556.556 INFO:teuthology.orchestra.console:Power off smithi002 2021-08-17T06:46:59.927 INFO:teuthology.kill:2021-08-17 06:46:59,926.926 INFO:teuthology.orchestra.console:Power off for smithi046 completed 2021-08-17T06:46:59.933 INFO:teuthology.kill:2021-08-17 06:46:59,932.932 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2021-08-17T06:47:00.243 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2021-08-17T06:47:00.244 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2021-08-17T06:47:00.244 INFO:teuthology.misc:Compressing logs... 2021-08-17T06:47:00.245 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-17T06:48:00.308 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-08-17T06:48:00.370 INFO:teuthology.kill:No teuthology processes running 2021-08-17T06:48:01.811 INFO:teuthology.kill:Nuking machines: ['smithi002', 'smithi046'] 2021-08-17T06:48:03.085 INFO:teuthology.kill:2021-08-17 06:48:03,084.084 INFO:teuthology.nuke:targets: 2021-08-17T06:48:03.086 INFO:teuthology.kill: smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBYFQCkfKXjojid/LA7aLMIhox/URnv+vOFL///TOBpMfMXnTrj+MR03ReLG8vh3tO9Jirm1mzpyUoVqamClaex6laov4CQNWS1+O2Qdt9qTOW8+zd0CIHaW94dTsqbfPzh6cQjT7U//kdj55HTQOXDTn8MCdmqKXoLY5q1gqXvMsf73a/yPZFDPsoh1XbQL8TXx0lXU4pObXQETknhJXVZWKmEUZPIM9OFCKlqWC+pNB/mzu9MaVBNoz7wcY6tyZhfQTcb8Mnd2Lf65Ul6ug+1SdJaXFJC3r4qJk1qGh0hP6g/nx/63hySt1KdD0m1cTD5qCMrtRqPC63B9RXnx5BviZ1iE9N0tfq2j9qaAADja2VkV8ClvWKxD+1OtLPK1MKGLpez4qAkGTr2uW55rAcWTvccrBoecauBejqr9Wf7ilGlrip5qeHtbMHVfOXRbtNiQExaR0U7zdSeizZQQVK9oOUylW3MGUT5xUcW1MXkNBK4IeFdAld7KD3iD13DLE= 2021-08-17T06:48:03.087 INFO:teuthology.kill: smithi046.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDU6wUZy/hc5n4i/RZ1RSbIHmipdIQPbpB8K7HfvvD3ChyYszcr2m1a8K2zU5wJ1SSFutR3g/5P6Ra0Bu13oObD6j6ZpP0zeD+p3gWf8x5L+rlgUyDiy20Vre3SxczHVGhtgDyIJBWgMfRLoZv7Rz6Hvy1VqbjvJnYSo4i+Tciw22uFIR9Gz7yaekM1lRcHKHvNwZlsWtUAysamS8QcURpMmw6qUAqQL4IXrbOhPDP5nJPQKfJ0Uf+LVnQkACELKJQZq/K0TUScE3QPOviznJ3uOGwp0hNsXN6kEM681n4RH/O+rCouE1K0ddRlbZZtW6cOdFwVTZ77kjSyNQXOBNACFsj7A0dMeXCOEAfPa8Hi+c8SqQlhNoP2wg0O7RPFddBtJuipG6vCgf/NnuOCzP3bbZ8zRIhXLUgyb0fP1/RRq7WbGwqc/Ij8mvr0Sq8LwWn/qLq1Om9khq2v4fd/Gv9QKY0G98foD5pbJQM9QdaS7bq7CgN2v6h7RRqSVTefKss= 2021-08-17T06:48:03.124 INFO:teuthology.kill:2021-08-17 06:48:03,116.116 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-17T06:48:03.124 INFO:teuthology.kill:2021-08-17 06:48:03,123.123 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-17T06:48:03.209 INFO:teuthology.kill:2021-08-17 06:48:03,208.208 INFO:teuthology.orchestra.console:Power off smithi002 2021-08-17T06:48:03.253 INFO:teuthology.kill:2021-08-17 06:48:03,252.252 INFO:teuthology.orchestra.console:Power off smithi046 2021-08-17T06:48:03.402 INFO:teuthology.kill:2021-08-17 06:48:03,401.401 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2021-08-17T06:48:03.410 INFO:teuthology.kill:2021-08-17 06:48:03,409.409 INFO:teuthology.orchestra.console:Power off for smithi046 completed 2021-08-17T06:48:03.580 INFO:teuthology.kill:2021-08-17 06:48:03,579.579 INFO:teuthology.lock.ops:unlocked smithi002.front.sepia.ceph.com 2021-08-17T06:48:03.587 INFO:teuthology.kill:2021-08-17 06:48:03,586.586 INFO:teuthology.lock.ops:unlocked smithi046.front.sepia.ceph.com 2021-08-17T06:50:04.003 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-08-17T06:50:04.097 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-08-17T06:50:04.097 INFO:teuthology.nuke:Checking targets against current locks 2021-08-17T06:50:04.186 INFO:teuthology.nuke:Not nuking smithi046.front.sepia.ceph.com because description doesn't match 2021-08-17T06:50:04.187 INFO:teuthology.nuke:Not nuking smithi002.front.sepia.ceph.com because description doesn't match