2021-02-15T15:50:33.978 INFO:root:teuthology version: 1.1.0-64d5f180 2021-02-15T15:50:34.108 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi131.front.sepia.ceph.com' reimaging 2021-02-15T15:50:34.108 INFO:teuthology.lock.ops:Updating [ubuntu@smithi131.front.sepia.ceph.com]: reset os type and version on server 2021-02-15T15:50:34.109 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2021-02-15T15:50:34.124 INFO:teuthology.lock.ops:Node 'ubuntu@smithi131.front.sepia.ceph.com' reimaging is complete 2021-02-15T15:50:34.124 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi162.front.sepia.ceph.com' reimaging 2021-02-15T15:50:34.124 INFO:teuthology.lock.ops:Updating [ubuntu@smithi162.front.sepia.ceph.com]: reset os type and version on server 2021-02-15T15:50:34.124 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2021-02-15T15:50:34.140 INFO:teuthology.lock.ops:Node 'ubuntu@smithi162.front.sepia.ceph.com' reimaging is complete 2021-02-15T15:50:34.328 INFO:teuthology.provision.fog.smithi131:Scheduling deploy of rhel 8.3 2021-02-15T15:50:34.345 INFO:teuthology.provision.fog.smithi162:Scheduling deploy of rhel 8.3 2021-02-15T15:50:34.668 INFO:teuthology.orchestra.console:Power off smithi162 2021-02-15T15:50:34.727 INFO:teuthology.orchestra.console:Power off smithi131 2021-02-15T15:50:43.089 INFO:teuthology.orchestra.console:Power off for smithi131 completed 2021-02-15T15:50:43.190 INFO:teuthology.orchestra.console:Power on smithi131 2021-02-15T15:50:47.147 INFO:teuthology.orchestra.console:Power off for smithi162 completed 2021-02-15T15:50:47.248 INFO:teuthology.orchestra.console:Power on smithi162 2021-02-15T15:50:51.574 INFO:teuthology.orchestra.console:Power on for smithi131 completed 2021-02-15T15:50:51.675 INFO:teuthology.provision.fog.smithi131:Waiting for deploy to finish 2021-02-15T15:50:55.612 INFO:teuthology.orchestra.console:Power on for smithi162 completed 2021-02-15T15:50:55.713 INFO:teuthology.provision.fog.smithi162:Waiting for deploy to finish 2021-02-15T15:55:14.233 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-15T15:55:14.488 INFO:teuthology.provision.fog.smithi131:Node is ready 2021-02-15T15:55:14.502 INFO:teuthology.orchestra.run.smithi131.stdout:smithi131.front.sepia.ceph.com 2021-02-15T15:55:14.557 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131 smithi131.front.sepia.ceph.com smithi131 2021-02-15T15:55:14.889 INFO:teuthology.provision.fog.smithi131:Deploy complete! 2021-02-15T15:55:36.455 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-15T15:55:36.772 INFO:teuthology.provision.fog.smithi162:Node is ready 2021-02-15T15:55:36.787 INFO:teuthology.orchestra.run.smithi162.stdout:smithi162.front.sepia.ceph.com 2021-02-15T15:55:36.842 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162 smithi162.front.sepia.ceph.com smithi162 2021-02-15T15:55:37.152 INFO:teuthology.provision.fog.smithi162:Deploy complete! 2021-02-15T15:55:37.467 INFO:teuthology.lock.ops:Checking smithi131.front.sepia.ceph.com 2021-02-15T15:55:37.468 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-15T15:55:37.484 INFO:teuthology.lock.ops:Checking smithi162.front.sepia.ceph.com 2021-02-15T15:55:37.484 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-15T15:55:37.500 INFO:teuthology.lock.ops:Updating [smithi131.front.sepia.ceph.com]: set os type and version on server 2021-02-15T15:55:37.500 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-15T15:55:37.788 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2021-02-15T15:55:37.842 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="Red Hat Enterprise Linux" 2021-02-15T15:55:37.842 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="8.3 (Ootpa)" 2021-02-15T15:55:37.843 INFO:teuthology.orchestra.run.smithi131.stdout:ID="rhel" 2021-02-15T15:55:37.843 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="fedora" 2021-02-15T15:55:37.843 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="8.3" 2021-02-15T15:55:37.843 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el8" 2021-02-15T15:55:37.843 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-02-15T15:55:37.843 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2021-02-15T15:55:37.843 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-02-15T15:55:37.843 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://www.redhat.com/" 2021-02-15T15:55:37.843 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-02-15T15:55:37.843 INFO:teuthology.orchestra.run.smithi131.stdout: 2021-02-15T15:55:37.843 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-02-15T15:55:37.844 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-02-15T15:55:37.844 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-02-15T15:55:37.844 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-02-15T15:55:37.845 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2021-02-15T15:55:37.860 INFO:teuthology.lock.ops:Updating [smithi162.front.sepia.ceph.com]: set os type and version on server 2021-02-15T15:55:37.860 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-15T15:55:38.147 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2021-02-15T15:55:38.201 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="Red Hat Enterprise Linux" 2021-02-15T15:55:38.201 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="8.3 (Ootpa)" 2021-02-15T15:55:38.201 INFO:teuthology.orchestra.run.smithi162.stdout:ID="rhel" 2021-02-15T15:55:38.201 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="fedora" 2021-02-15T15:55:38.201 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="8.3" 2021-02-15T15:55:38.201 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el8" 2021-02-15T15:55:38.201 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-02-15T15:55:38.201 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2021-02-15T15:55:38.202 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-02-15T15:55:38.202 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://www.redhat.com/" 2021-02-15T15:55:38.202 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-02-15T15:55:38.202 INFO:teuthology.orchestra.run.smithi162.stdout: 2021-02-15T15:55:38.202 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-02-15T15:55:38.202 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-02-15T15:55:38.202 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-02-15T15:55:38.202 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-02-15T15:55:38.203 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2021-02-15T15:55:38.283 INFO:teuthology.dispatcher.supervisor:Running job 5884254 2021-02-15T15:55:38.283 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/virtualenv/bin/teuthology -v --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2021-02-15_15:21:14-rados-wip-sage2-testing-2021-02-10-1604-distro-basic-smithi/5884254 --name sage-2021-02-15_15:21:14-rados-wip-sage2-testing-2021-02-10-1604-distro-basic-smithi --description rados/cephadm/thrash/{0-distro/rhel_latest 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} -- /home/teuthworker/archive/sage-2021-02-15_15:21:14-rados-wip-sage2-testing-2021-02-10-1604-distro-basic-smithi/5884254/orig.config.yaml 2021-02-15T15:55:38.298 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sage-2021-02-15_15:21:14-rados-wip-sage2-testing-2021-02-10-1604-distro-basic-smithi/5884254 2021-02-15T15:55:38.299 INFO:teuthology.dispatcher.supervisor:Job PID: 15516 2021-02-15T15:55:38.299 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-02-16T03:55:54.677 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-02-16T03:55:55.313 INFO:teuthology.kill:Killing Pids: {15516} 2021-02-16T03:55:57.238 INFO:teuthology.kill:Nuking machines: ['smithi131', 'smithi162'] 2021-02-16T03:55:58.582 INFO:teuthology.kill:2021-02-16 03:55:58,581.581 INFO:teuthology.nuke:targets: 2021-02-16T03:55:58.583 INFO:teuthology.kill: smithi131.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0T63C78kUZCf8ZevSEL8ZM7lfR2JN0v515gNJp7vVzjYPi6qkQiDl94C9Uf1DMbhGz/iDJT6JAq5mNP0Zldaii9PdGlNID7zCdkRfHK+7O2EdNwAUTzUN3uckERuyyqGJ61/aph0tnBaqS4uK93z30LEFg/ICp4+c3MifMv0y1Q8Gtc5o445EEUXzDFT7jU9nVNLpExj4mdjigdH6d8EiLluGvHc07ucTr2dOyloIZUUPIsv1p5OVk2emZwpr4fZG/b1uCzBHpmd+XJsBjGyvqWSjVdaelZlR5rk1tk/hhYsdhEIVy0nTeAkNgZhL5COCndpa2f1myujqMfZPVDEblkT8fZPXBEpdnebupZfMBMYNXpHJeTgAhQOwiWE2sT34LgXnAKzquye3J+mLl+58nq5oYtqScp6rzO65hELhLigET1gZglExDuR78H+oCEwtGqF7EqAJrYIfcg5JopnbBFeZAFQCW3FZyGSHFfeJEbrVSKyPuTRabTT9JNx3zzs= 2021-02-16T03:55:58.583 INFO:teuthology.kill: smithi162.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbpzyHGNpeWrmfsRSj6edp/F5HgGHqDBgjWYWBqn/6v35V0FTyphZu8+PdTV4ZIT065+2k3x589zLgnz+S6Vb2Fs+G3dubNp7ggg9quFxz3jAzp18zRdRW24fP/D8RT93i/JR+X9lcf9jJMuJelhtwofmlhM+MYydTXIYhOjZHbu3xe8D/qdMUpErk3nP/QCWa0cImaGlXxyBs0bp1c6rMbqX3WVDeDmsytGnAwRalOpRJ1tfK0vjDxvHkcLkHcGWRGFjdtw+Hjju7Ybq6s6MoOQ+LZlR8KsMsBRbsNdWMZYjI2sxT44gBLcNNM2/+2TPBn9zutf9yA/I3/4LXZ0zdG8BkGmScyqBSg34piuiSpoWnw7hxvFrytVmjki6DaOttZ/tci7E7h5/NFJpuCXVW3sMldx9aIhb+LTmtPb8hD1KFqF7qSnH8FELoutpSzhY/hws1kRai1HCMtnGJjN7+tXW43EHgSNmvRv82RwOB1oXid91YU6dwEp047BAM94s= 2021-02-16T03:55:58.583 INFO:teuthology.kill:2021-02-16 03:55:58,582.582 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-16T03:55:58.592 INFO:teuthology.kill:2021-02-16 03:55:58,592.592 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-16T03:55:58.660 INFO:teuthology.kill:2021-02-16 03:55:58,660.660 INFO:teuthology.orchestra.console:Power off smithi162 2021-02-16T03:55:58.692 INFO:teuthology.kill:2021-02-16 03:55:58,692.692 INFO:teuthology.orchestra.console:Power off smithi131 2021-02-16T03:56:07.097 INFO:teuthology.kill:2021-02-16 03:56:07,096.096 INFO:teuthology.orchestra.console:Power off for smithi162 completed 2021-02-16T03:56:11.199 INFO:teuthology.kill:2021-02-16 03:56:11,199.199 INFO:teuthology.orchestra.console:Power off for smithi131 completed 2021-02-16T03:56:11.671 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.realm.zone.a', 'node-exporter.a', 'alertmanager.a'] 2021-02-16T03:56:11.672 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2021-02-16T03:56:11.675 INFO:teuthology.misc:Compressing logs... 2021-02-16T03:56:11.684 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-16T03:56:46.415 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/misc.py", line 1372, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-02-16T03:56:46.505 INFO:teuthology.kill:No teuthology processes running 2021-02-16T03:56:47.507 INFO:teuthology.kill:Nuking machines: ['smithi131', 'smithi162'] 2021-02-16T03:56:48.730 INFO:teuthology.kill:2021-02-16 03:56:48,730.730 INFO:teuthology.nuke:targets: 2021-02-16T03:56:48.730 INFO:teuthology.kill: smithi131.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0T63C78kUZCf8ZevSEL8ZM7lfR2JN0v515gNJp7vVzjYPi6qkQiDl94C9Uf1DMbhGz/iDJT6JAq5mNP0Zldaii9PdGlNID7zCdkRfHK+7O2EdNwAUTzUN3uckERuyyqGJ61/aph0tnBaqS4uK93z30LEFg/ICp4+c3MifMv0y1Q8Gtc5o445EEUXzDFT7jU9nVNLpExj4mdjigdH6d8EiLluGvHc07ucTr2dOyloIZUUPIsv1p5OVk2emZwpr4fZG/b1uCzBHpmd+XJsBjGyvqWSjVdaelZlR5rk1tk/hhYsdhEIVy0nTeAkNgZhL5COCndpa2f1myujqMfZPVDEblkT8fZPXBEpdnebupZfMBMYNXpHJeTgAhQOwiWE2sT34LgXnAKzquye3J+mLl+58nq5oYtqScp6rzO65hELhLigET1gZglExDuR78H+oCEwtGqF7EqAJrYIfcg5JopnbBFeZAFQCW3FZyGSHFfeJEbrVSKyPuTRabTT9JNx3zzs= 2021-02-16T03:56:48.731 INFO:teuthology.kill: smithi162.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbpzyHGNpeWrmfsRSj6edp/F5HgGHqDBgjWYWBqn/6v35V0FTyphZu8+PdTV4ZIT065+2k3x589zLgnz+S6Vb2Fs+G3dubNp7ggg9quFxz3jAzp18zRdRW24fP/D8RT93i/JR+X9lcf9jJMuJelhtwofmlhM+MYydTXIYhOjZHbu3xe8D/qdMUpErk3nP/QCWa0cImaGlXxyBs0bp1c6rMbqX3WVDeDmsytGnAwRalOpRJ1tfK0vjDxvHkcLkHcGWRGFjdtw+Hjju7Ybq6s6MoOQ+LZlR8KsMsBRbsNdWMZYjI2sxT44gBLcNNM2/+2TPBn9zutf9yA/I3/4LXZ0zdG8BkGmScyqBSg34piuiSpoWnw7hxvFrytVmjki6DaOttZ/tci7E7h5/NFJpuCXVW3sMldx9aIhb+LTmtPb8hD1KFqF7qSnH8FELoutpSzhY/hws1kRai1HCMtnGJjN7+tXW43EHgSNmvRv82RwOB1oXid91YU6dwEp047BAM94s= 2021-02-16T03:56:48.754 INFO:teuthology.kill:2021-02-16 03:56:48,754.754 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-16T03:56:48.757 INFO:teuthology.kill:2021-02-16 03:56:48,757.757 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-16T03:56:48.810 INFO:teuthology.kill:2021-02-16 03:56:48,810.810 INFO:teuthology.orchestra.console:Power off smithi162 2021-02-16T03:56:48.837 INFO:teuthology.kill:2021-02-16 03:56:48,836.836 INFO:teuthology.orchestra.console:Power off smithi131 2021-02-16T03:56:48.990 INFO:teuthology.kill:2021-02-16 03:56:48,989.989 INFO:teuthology.orchestra.console:Power off for smithi162 completed 2021-02-16T03:56:48.994 INFO:teuthology.kill:2021-02-16 03:56:48,994.994 INFO:teuthology.orchestra.console:Power off for smithi131 completed 2021-02-16T03:56:49.137 INFO:teuthology.kill:2021-02-16 03:56:49,136.136 INFO:teuthology.lock.ops:unlocked smithi162.front.sepia.ceph.com 2021-02-16T03:56:49.140 INFO:teuthology.kill:2021-02-16 03:56:49,140.140 INFO:teuthology.lock.ops:unlocked smithi131.front.sepia.ceph.com 2021-02-16T03:58:49.319 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15