2021-02-12T23:15:44.069 INFO:root:teuthology version: 1.1.0-9ec73f3a 2021-02-12T23:15:44.196 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi114.front.sepia.ceph.com' reimaging 2021-02-12T23:15:44.197 INFO:teuthology.lock.ops:Updating [ubuntu@smithi114.front.sepia.ceph.com]: reset os type and version on server 2021-02-12T23:15:44.197 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2021-02-12T23:15:44.211 INFO:teuthology.lock.ops:Node 'ubuntu@smithi114.front.sepia.ceph.com' reimaging is complete 2021-02-12T23:15:44.211 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi176.front.sepia.ceph.com' reimaging 2021-02-12T23:15:44.211 INFO:teuthology.lock.ops:Updating [ubuntu@smithi176.front.sepia.ceph.com]: reset os type and version on server 2021-02-12T23:15:44.212 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2021-02-12T23:15:44.229 INFO:teuthology.lock.ops:Node 'ubuntu@smithi176.front.sepia.ceph.com' reimaging is complete 2021-02-12T23:15:44.405 INFO:teuthology.provision.fog.smithi114:Scheduling deploy of rhel 8.3 2021-02-12T23:15:44.441 INFO:teuthology.provision.fog.smithi176:Scheduling deploy of rhel 8.3 2021-02-12T23:15:44.791 INFO:teuthology.orchestra.console:Power off smithi114 2021-02-12T23:15:44.839 INFO:teuthology.orchestra.console:Power off smithi176 2021-02-12T23:15:57.302 INFO:teuthology.orchestra.console:Power off for smithi114 completed 2021-02-12T23:15:57.325 INFO:teuthology.orchestra.console:Power off for smithi176 completed 2021-02-12T23:15:57.404 INFO:teuthology.orchestra.console:Power on smithi114 2021-02-12T23:15:57.426 INFO:teuthology.orchestra.console:Power on smithi176 2021-02-12T23:16:05.802 INFO:teuthology.orchestra.console:Power on for smithi176 completed 2021-02-12T23:16:05.814 INFO:teuthology.orchestra.console:Power on for smithi114 completed 2021-02-12T23:16:05.904 INFO:teuthology.provision.fog.smithi176:Waiting for deploy to finish 2021-02-12T23:16:05.915 INFO:teuthology.provision.fog.smithi114:Waiting for deploy to finish 2021-02-12T23:20:37.865 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-12T23:20:37.933 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-12T23:20:38.177 INFO:teuthology.provision.fog.smithi114:Node is ready 2021-02-12T23:20:38.192 INFO:teuthology.orchestra.run.smithi114.stdout:smithi114.front.sepia.ceph.com 2021-02-12T23:20:38.232 INFO:teuthology.provision.fog.smithi176:Node is ready 2021-02-12T23:20:38.246 INFO:teuthology.orchestra.run.smithi176.stdout:smithi176.front.sepia.ceph.com 2021-02-12T23:20:38.247 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114 smithi114.front.sepia.ceph.com smithi114 2021-02-12T23:20:38.301 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176 smithi176.front.sepia.ceph.com smithi176 2021-02-12T23:20:38.583 INFO:teuthology.provision.fog.smithi114:Deploy complete! 2021-02-12T23:20:38.627 INFO:teuthology.provision.fog.smithi176:Deploy complete! 2021-02-12T23:20:38.761 INFO:teuthology.lock.ops:Checking smithi114.front.sepia.ceph.com 2021-02-12T23:20:38.761 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-12T23:20:38.778 INFO:teuthology.lock.ops:Checking smithi176.front.sepia.ceph.com 2021-02-12T23:20:38.778 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-12T23:20:38.793 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: set os type and version on server 2021-02-12T23:20:38.793 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-12T23:20:39.113 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2021-02-12T23:20:39.168 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="Red Hat Enterprise Linux" 2021-02-12T23:20:39.168 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="8.3 (Ootpa)" 2021-02-12T23:20:39.169 INFO:teuthology.orchestra.run.smithi114.stdout:ID="rhel" 2021-02-12T23:20:39.169 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="fedora" 2021-02-12T23:20:39.169 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="8.3" 2021-02-12T23:20:39.169 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el8" 2021-02-12T23:20:39.169 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-02-12T23:20:39.169 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2021-02-12T23:20:39.170 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-02-12T23:20:39.170 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://www.redhat.com/" 2021-02-12T23:20:39.170 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-02-12T23:20:39.170 INFO:teuthology.orchestra.run.smithi114.stdout: 2021-02-12T23:20:39.170 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-02-12T23:20:39.171 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-02-12T23:20:39.171 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-02-12T23:20:39.171 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-02-12T23:20:39.173 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2021-02-12T23:20:39.190 INFO:teuthology.lock.ops:Updating [smithi176.front.sepia.ceph.com]: set os type and version on server 2021-02-12T23:20:39.190 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-12T23:20:39.470 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2021-02-12T23:20:39.525 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="Red Hat Enterprise Linux" 2021-02-12T23:20:39.525 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="8.3 (Ootpa)" 2021-02-12T23:20:39.525 INFO:teuthology.orchestra.run.smithi176.stdout:ID="rhel" 2021-02-12T23:20:39.525 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE="fedora" 2021-02-12T23:20:39.525 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="8.3" 2021-02-12T23:20:39.525 INFO:teuthology.orchestra.run.smithi176.stdout:PLATFORM_ID="platform:el8" 2021-02-12T23:20:39.526 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-02-12T23:20:39.526 INFO:teuthology.orchestra.run.smithi176.stdout:ANSI_COLOR="0;31" 2021-02-12T23:20:39.526 INFO:teuthology.orchestra.run.smithi176.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-02-12T23:20:39.526 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://www.redhat.com/" 2021-02-12T23:20:39.526 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-02-12T23:20:39.527 INFO:teuthology.orchestra.run.smithi176.stdout: 2021-02-12T23:20:39.527 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-02-12T23:20:39.527 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-02-12T23:20:39.527 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-02-12T23:20:39.527 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-02-12T23:20:39.528 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2021-02-12T23:20:39.605 INFO:teuthology.dispatcher.supervisor:Running job 5874250 2021-02-12T23:20:39.605 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_9ec73f3aa0a18603afa55289d00653272f224a69/virtualenv/bin/teuthology -v --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2021-02-10_23:48:41-rados-wip-sage2-testing-2021-02-10-1604-distro-basic-smithi/5874250 --name sage-2021-02-10_23:48:41-rados-wip-sage2-testing-2021-02-10-1604-distro-basic-smithi --description rados/cephadm/thrash/{0-distro/rhel_latest 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} -- /home/teuthworker/archive/sage-2021-02-10_23:48:41-rados-wip-sage2-testing-2021-02-10-1604-distro-basic-smithi/5874250/orig.config.yaml 2021-02-12T23:20:39.611 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sage-2021-02-10_23:48:41-rados-wip-sage2-testing-2021-02-10-1604-distro-basic-smithi/5874250 2021-02-12T23:20:39.611 INFO:teuthology.dispatcher.supervisor:Job PID: 31137 2021-02-12T23:20:39.611 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-02-13T11:20:55.691 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-02-13T11:20:55.977 INFO:teuthology.kill:Killing Pids: {31137} 2021-02-13T11:20:56.886 INFO:teuthology.kill:Nuking machines: ['smithi114', 'smithi176'] 2021-02-13T11:20:57.640 INFO:teuthology.kill:2021-02-13 11:20:57,639.639 INFO:teuthology.nuke:targets: 2021-02-13T11:20:57.640 INFO:teuthology.kill: smithi114.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXYbjG4BOPPIp48dBoYKNZ7eQEJUYtdtjYDWdIkHhbTjHqyajOPXMYTZlu+K8UKn8IWX6n1qLeJ/lGehEYmJDORhSe0Hl+hzlNfDpMZwBvIiQ42q6Oo7wdAZNb2auygS1RcMPZeYKwHiXPUUWSCpHprX8dI91ha9rEz3deqsLrTKZ9FPMSep2lYwzsLf9hfZvgmKeE7/ODmoaw5zwLiUrkPUT787GHuOFwT38raQvbqdGaoJKnBunAjwfJj+1Vbut20HmwVpytzvzuL86Z3W8GP4yVjRA41YS6C/ypu/o/VUHnvCTwasEd2N1AKBWmrJwzVUOh+lTCmx2WlH/0f5Ho0tEXOjvW55bn3RdPdha4YrVdrGfsCHxhHviW6f2caOSAh4ZL2sjc7zRyq2KM58p2gn2KlYi7yEL3QsWaSPiLIRoHbMMcS8p8KkcjhUmtv5oAd6bvh2HrrG1FK4tXtmKZrgZ5DAv8RGVbWju55jO/WkiIz+TJzDQz1Q2Kq8hEyaE= 2021-02-13T11:20:57.640 INFO:teuthology.kill: smithi176.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDG53Vrgl7W21GZ/Ptrnv+aHeoMhFPLOwyaYw5fo1wTPv+qgU7hJD7/iFp3fP9ti8fhizUX51EiXM3tFaciM04RoRSTXkH+GSRPqAq+rN3jsFnnYiFOxKLGCiTIK8n+40EogaM0yGqo9LYwF9h4BgBTaZ044junpifV48NJGMiUNWLUpwICEYFk1NezPfQhWeBzNeR7YKgHxSg78JE6pXTr5OBm7TsNyrutfFCQ06twJLQec2lXL1K5vxJbmIP5q/E7QriKacdnO9bLCUfRXIlyr2L+mjJ2Nk5Y5W5WFao9pGLxPL7s181JYtaAQGUFjlN6hNgWkay1yI4KdiMMqn1VzuciMVeZCsMQEm2mMorWK9YjfgtYQWc3KqGQxA3JEkVXTS0olxN8EwNiEiKye8RNGYbnvUHK2waoSaH5FzH8uy9QuymZVg6dv6TFtx3N5XShuBDPID33LQBBO2dGOu4Yv6S1bncMYokZX9re/ddUALTpdVCfxj7Ln6XIlhkxcR0= 2021-02-13T11:20:57.640 INFO:teuthology.kill:2021-02-13 11:20:57,640.640 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-13T11:20:57.645 INFO:teuthology.kill:2021-02-13 11:20:57,645.645 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-13T11:20:57.704 INFO:teuthology.kill:2021-02-13 11:20:57,703.703 INFO:teuthology.orchestra.console:Power off smithi176 2021-02-13T11:20:57.715 INFO:teuthology.kill:2021-02-13 11:20:57,715.715 INFO:teuthology.orchestra.console:Power off smithi114 2021-02-13T11:21:10.186 INFO:teuthology.kill:2021-02-13 11:21:10,185.185 INFO:teuthology.orchestra.console:Power off for smithi176 completed 2021-02-13T11:21:10.192 INFO:teuthology.kill:2021-02-13 11:21:10,192.192 INFO:teuthology.orchestra.console:Power off for smithi114 completed 2021-02-13T11:21:10.411 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.realm.zone.a', 'node-exporter.a', 'alertmanager.a'] 2021-02-13T11:21:10.412 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2021-02-13T11:21:10.412 INFO:teuthology.misc:Compressing logs... 2021-02-13T11:21:10.412 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-13T11:21:44.898 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_9ec73f3aa0a18603afa55289d00653272f224a69/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_9ec73f3aa0a18603afa55289d00653272f224a69/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_9ec73f3aa0a18603afa55289d00653272f224a69/teuthology/misc.py", line 1372, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_9ec73f3aa0a18603afa55289d00653272f224a69/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_9ec73f3aa0a18603afa55289d00653272f224a69/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_9ec73f3aa0a18603afa55289d00653272f224a69/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_9ec73f3aa0a18603afa55289d00653272f224a69/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-02-13T11:21:44.947 INFO:teuthology.kill:No teuthology processes running 2021-02-13T11:21:45.799 INFO:teuthology.kill:Nuking machines: ['smithi114', 'smithi176'] 2021-02-13T11:21:46.547 INFO:teuthology.kill:2021-02-13 11:21:46,546.546 INFO:teuthology.nuke:targets: 2021-02-13T11:21:46.547 INFO:teuthology.kill: smithi114.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXYbjG4BOPPIp48dBoYKNZ7eQEJUYtdtjYDWdIkHhbTjHqyajOPXMYTZlu+K8UKn8IWX6n1qLeJ/lGehEYmJDORhSe0Hl+hzlNfDpMZwBvIiQ42q6Oo7wdAZNb2auygS1RcMPZeYKwHiXPUUWSCpHprX8dI91ha9rEz3deqsLrTKZ9FPMSep2lYwzsLf9hfZvgmKeE7/ODmoaw5zwLiUrkPUT787GHuOFwT38raQvbqdGaoJKnBunAjwfJj+1Vbut20HmwVpytzvzuL86Z3W8GP4yVjRA41YS6C/ypu/o/VUHnvCTwasEd2N1AKBWmrJwzVUOh+lTCmx2WlH/0f5Ho0tEXOjvW55bn3RdPdha4YrVdrGfsCHxhHviW6f2caOSAh4ZL2sjc7zRyq2KM58p2gn2KlYi7yEL3QsWaSPiLIRoHbMMcS8p8KkcjhUmtv5oAd6bvh2HrrG1FK4tXtmKZrgZ5DAv8RGVbWju55jO/WkiIz+TJzDQz1Q2Kq8hEyaE= 2021-02-13T11:21:46.547 INFO:teuthology.kill: smithi176.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDG53Vrgl7W21GZ/Ptrnv+aHeoMhFPLOwyaYw5fo1wTPv+qgU7hJD7/iFp3fP9ti8fhizUX51EiXM3tFaciM04RoRSTXkH+GSRPqAq+rN3jsFnnYiFOxKLGCiTIK8n+40EogaM0yGqo9LYwF9h4BgBTaZ044junpifV48NJGMiUNWLUpwICEYFk1NezPfQhWeBzNeR7YKgHxSg78JE6pXTr5OBm7TsNyrutfFCQ06twJLQec2lXL1K5vxJbmIP5q/E7QriKacdnO9bLCUfRXIlyr2L+mjJ2Nk5Y5W5WFao9pGLxPL7s181JYtaAQGUFjlN6hNgWkay1yI4KdiMMqn1VzuciMVeZCsMQEm2mMorWK9YjfgtYQWc3KqGQxA3JEkVXTS0olxN8EwNiEiKye8RNGYbnvUHK2waoSaH5FzH8uy9QuymZVg6dv6TFtx3N5XShuBDPID33LQBBO2dGOu4Yv6S1bncMYokZX9re/ddUALTpdVCfxj7Ln6XIlhkxcR0= 2021-02-13T11:21:46.569 INFO:teuthology.kill:2021-02-13 11:21:46,569.569 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-13T11:21:46.573 INFO:teuthology.kill:2021-02-13 11:21:46,573.573 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-13T11:21:46.628 INFO:teuthology.kill:2021-02-13 11:21:46,628.628 INFO:teuthology.orchestra.console:Power off smithi114 2021-02-13T11:21:46.641 INFO:teuthology.kill:2021-02-13 11:21:46,641.641 INFO:teuthology.orchestra.console:Power off smithi176 2021-02-13T11:21:46.769 INFO:teuthology.kill:2021-02-13 11:21:46,769.769 INFO:teuthology.orchestra.console:Power off for smithi114 completed 2021-02-13T11:21:46.780 INFO:teuthology.kill:2021-02-13 11:21:46,780.780 INFO:teuthology.orchestra.console:Power off for smithi176 completed 2021-02-13T11:21:46.912 INFO:teuthology.kill:2021-02-13 11:21:46,911.911 INFO:teuthology.lock.ops:unlocked smithi114.front.sepia.ceph.com 2021-02-13T11:21:46.919 INFO:teuthology.kill:2021-02-13 11:21:46,919.919 INFO:teuthology.lock.ops:unlocked smithi176.front.sepia.ceph.com 2021-02-13T11:23:47.089 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-02-13T11:23:47.168 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-02-13T11:23:47.168 INFO:teuthology.nuke:Checking targets against current locks 2021-02-13T11:23:47.244 INFO:teuthology.nuke:Not nuking smithi114.front.sepia.ceph.com because description doesn't match 2021-02-13T11:23:47.245 INFO:teuthology.nuke:Not nuking smithi176.front.sepia.ceph.com because description doesn't match