2021-02-11T00:25:33.093 INFO:root:teuthology version: 1.1.0-14fad317 2021-02-11T00:25:33.198 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi091.front.sepia.ceph.com' reimaging 2021-02-11T00:25:33.198 INFO:teuthology.lock.ops:Updating [ubuntu@smithi091.front.sepia.ceph.com]: reset os type and version on server 2021-02-11T00:25:33.198 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server 2021-02-11T00:25:33.213 INFO:teuthology.lock.ops:Node 'ubuntu@smithi091.front.sepia.ceph.com' reimaging is complete 2021-02-11T00:25:33.213 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging 2021-02-11T00:25:33.213 INFO:teuthology.lock.ops:Updating [ubuntu@smithi110.front.sepia.ceph.com]: reset os type and version on server 2021-02-11T00:25:33.214 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2021-02-11T00:25:33.228 INFO:teuthology.lock.ops:Node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging is complete 2021-02-11T00:25:33.413 INFO:teuthology.provision.fog.smithi091:Scheduling deploy of centos 8.0 2021-02-11T00:25:33.423 INFO:teuthology.provision.fog.smithi110:Scheduling deploy of centos 8.0 2021-02-11T00:25:33.751 INFO:teuthology.orchestra.console:Power off smithi091 2021-02-11T00:25:33.865 INFO:teuthology.orchestra.console:Power off smithi110 2021-02-11T00:25:42.256 INFO:teuthology.orchestra.console:Power off for smithi091 completed 2021-02-11T00:25:42.357 INFO:teuthology.orchestra.console:Power on smithi091 2021-02-11T00:25:46.359 INFO:teuthology.orchestra.console:Power off for smithi110 completed 2021-02-11T00:25:46.460 INFO:teuthology.orchestra.console:Power on smithi110 2021-02-11T00:25:50.720 INFO:teuthology.orchestra.console:Power on for smithi110 completed 2021-02-11T00:25:50.724 INFO:teuthology.orchestra.console:Power on for smithi091 completed 2021-02-11T00:25:50.821 INFO:teuthology.provision.fog.smithi110:Waiting for deploy to finish 2021-02-11T00:25:50.825 INFO:teuthology.provision.fog.smithi091:Waiting for deploy to finish 2021-02-11T00:30:00.721 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-11T00:30:19.981 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-11T00:30:20.305 INFO:teuthology.provision.fog.smithi091:Node is ready 2021-02-11T00:30:20.359 INFO:teuthology.orchestra.run.smithi091.stdout:smithi091.front.sepia.ceph.com 2021-02-11T00:30:20.413 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.15.91 smithi091.front.sepia.ceph.com smithi091 2021-02-11T00:30:20.763 INFO:teuthology.provision.fog.smithi091:Deploy complete! 2021-02-11T00:30:21.023 INFO:teuthology.provision.fog.smithi110:Node is ready 2021-02-11T00:30:21.041 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110.front.sepia.ceph.com 2021-02-11T00:30:21.097 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110 smithi110.front.sepia.ceph.com smithi110 2021-02-11T00:30:21.473 INFO:teuthology.provision.fog.smithi110:Deploy complete! 2021-02-11T00:30:21.597 INFO:teuthology.lock.ops:Checking smithi091.front.sepia.ceph.com 2021-02-11T00:30:21.598 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-11T00:30:21.614 INFO:teuthology.lock.ops:Checking smithi110.front.sepia.ceph.com 2021-02-11T00:30:21.614 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-11T00:30:21.628 INFO:teuthology.lock.ops:Updating [smithi091.front.sepia.ceph.com]: set os type and version on server 2021-02-11T00:30:21.629 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-11T00:30:21.909 INFO:teuthology.orchestra.run.smithi091.stdout:x86_64 2021-02-11T00:30:21.963 INFO:teuthology.orchestra.run.smithi091.stdout:NAME="CentOS Linux" 2021-02-11T00:30:21.963 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION="8 (Core)" 2021-02-11T00:30:21.963 INFO:teuthology.orchestra.run.smithi091.stdout:ID="centos" 2021-02-11T00:30:21.963 INFO:teuthology.orchestra.run.smithi091.stdout:ID_LIKE="rhel fedora" 2021-02-11T00:30:21.963 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_ID="8" 2021-02-11T00:30:21.963 INFO:teuthology.orchestra.run.smithi091.stdout:PLATFORM_ID="platform:el8" 2021-02-11T00:30:21.963 INFO:teuthology.orchestra.run.smithi091.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-02-11T00:30:21.963 INFO:teuthology.orchestra.run.smithi091.stdout:ANSI_COLOR="0;31" 2021-02-11T00:30:21.963 INFO:teuthology.orchestra.run.smithi091.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-02-11T00:30:21.963 INFO:teuthology.orchestra.run.smithi091.stdout:HOME_URL="https://www.centos.org/" 2021-02-11T00:30:21.963 INFO:teuthology.orchestra.run.smithi091.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-02-11T00:30:21.964 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-02-11T00:30:21.964 INFO:teuthology.orchestra.run.smithi091.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-02-11T00:30:21.964 INFO:teuthology.orchestra.run.smithi091.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-02-11T00:30:21.964 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-02-11T00:30:21.964 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-02-11T00:30:21.964 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-02-11T00:30:21.965 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server 2021-02-11T00:30:21.981 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: set os type and version on server 2021-02-11T00:30:21.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-11T00:30:22.265 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2021-02-11T00:30:22.317 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Linux" 2021-02-11T00:30:22.318 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="8 (Core)" 2021-02-11T00:30:22.318 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2021-02-11T00:30:22.318 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2021-02-11T00:30:22.318 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="8" 2021-02-11T00:30:22.318 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el8" 2021-02-11T00:30:22.318 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-02-11T00:30:22.318 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2021-02-11T00:30:22.318 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-02-11T00:30:22.318 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://www.centos.org/" 2021-02-11T00:30:22.318 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-02-11T00:30:22.318 INFO:teuthology.orchestra.run.smithi110.stdout: 2021-02-11T00:30:22.319 INFO:teuthology.orchestra.run.smithi110.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-02-11T00:30:22.319 INFO:teuthology.orchestra.run.smithi110.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-02-11T00:30:22.319 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-02-11T00:30:22.319 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-02-11T00:30:22.319 INFO:teuthology.orchestra.run.smithi110.stdout: 2021-02-11T00:30:22.319 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2021-02-11T00:30:22.398 INFO:teuthology.dispatcher.supervisor:Running job 5873971 2021-02-11T00:30:22.398 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/virtualenv/bin/teuthology -v --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2021-02-10_23:47:44-rados:cephadm:thrash-wip-sage2-testing-2021-02-10-1604-distro-basic-smithi/5873971 --name sage-2021-02-10_23:47:44-rados:cephadm:thrash-wip-sage2-testing-2021-02-10-1604-distro-basic-smithi --description rados:cephadm:thrash/{0-distro/centos_8.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} -- /home/teuthworker/archive/sage-2021-02-10_23:47:44-rados:cephadm:thrash-wip-sage2-testing-2021-02-10-1604-distro-basic-smithi/5873971/orig.config.yaml 2021-02-11T00:30:22.402 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sage-2021-02-10_23:47:44-rados:cephadm:thrash-wip-sage2-testing-2021-02-10-1604-distro-basic-smithi/5873971 2021-02-11T00:30:22.403 INFO:teuthology.dispatcher.supervisor:Job PID: 25788 2021-02-11T00:30:22.403 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-02-11T12:30:38.007 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-02-11T12:30:54.474 INFO:teuthology.kill:Killing Pids: {25788} 2021-02-11T12:31:46.714 INFO:teuthology.kill:Nuking machines: ['smithi091', 'smithi110'] 2021-02-11T12:31:47.812 INFO:teuthology.kill:2021-02-11 12:31:47,811.811 INFO:teuthology.nuke:targets: 2021-02-11T12:31:47.812 INFO:teuthology.kill: smithi091.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXivQZ5qk4nWSchy1vnjW3yJ6rnEYQUVXhOHmoWzz4mMqCzYlXRDKTWgi7InVyLC9YnUXKz2TeeNqkDCJAFfnVTITLHwmgrOsmZnie1rtwr4mnPw7Envxx/+v+YLihqqljx58LdjvgYHtvJpRP4r1jpFOqeuTEyntawzrBScY0MMb82USaRd2CUOV7EI95uGJtDHuATTMAnF2kPiAACXF9pHJ4IbkjU+ZhBos7Ro7LLPDf8/9qZZIte5xRSX5WUW1vIpykRN8SBWfIPrkiieCmp9vudmAmsfTMFgPUrgRXplv8KVOw+hd/PTu2u6ty6jZIOGW9S7USgfWRLmG9IVcHdPBZJWORUS1h3fUcxURO1VzdOIK9aDI06z0ar2PdpUHaKphKL8As+0FmFdrQRaQ5r3h3Rq6zh2IOEU1ggpVqSYJ0IFi/BgOPWYAFJtBv+LxisJsDcx16DH15S466/Nn5EPa6ih45LArr79LIZu3FoIDSCn6ieoqu5MS1bIylkEE= 2021-02-11T12:31:47.812 INFO:teuthology.kill: smithi110.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFxEw4RkOqPutpdvA/M6ZM2A7frXxrH4Ouib8CCIOuMPf0tkZGDxx05Cof3+fV/7rflDqC1xlTH2PiLgI4cgerxP1I5FGMbpr/yTke/bNWhjq+/7Bu+pxuhP6G8i4BpJ/iGYaezVcwrb/EIImcKFFrZ5WdHfuc/0nmSFvBnHm5fnNVm1d1y6WkbeHtugEbfEQt0XIJjQskBiNGUDnFet6EIVOcFnLCG6V9DPtfAZqbUrosHZm/abhet2aaAe/I/0wv1Q8AAQj8w0rrmelcOUl4DgioeVEPHYZ1/twAzR1jOjqVpJ2JYkqr4AAIGOFBiEvJ1UDwio8/SdmN6wsjmO8fr5d7OBW5n3naGKiFuJNBwjac4niGswNFLxgZGwpd0oyqX7ViOP5GrBteBoSzGOEWiHJGo/IasPaOx2vBMLA1Rz1JLESqOzz4hS8XrZdPb/ohgio2p9JGeeqWqeaXiovI7RU+w3S1PA/+GTm1AMXnM/lm8t4xxHNtxthqonTCWkk= 2021-02-11T12:31:47.812 INFO:teuthology.kill:2021-02-11 12:31:47,812.812 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-11T12:31:47.818 INFO:teuthology.kill:2021-02-11 12:31:47,817.817 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-11T12:31:47.878 INFO:teuthology.kill:2021-02-11 12:31:47,878.878 INFO:teuthology.orchestra.console:Power off smithi091 2021-02-11T12:31:47.896 INFO:teuthology.kill:2021-02-11 12:31:47,895.895 INFO:teuthology.orchestra.console:Power off smithi110 2021-02-11T12:32:00.388 INFO:teuthology.kill:2021-02-11 12:32:00,387.387 INFO:teuthology.orchestra.console:Power off for smithi091 completed 2021-02-11T12:32:43.683 INFO:teuthology.kill:2021-02-11 12:32:00,394.394 INFO:teuthology.orchestra.console:Power off for smithi110 completed 2021-02-11T12:32:43.862 INFO:teuthology.task.internal:roles: ubuntu@smithi091.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.realm.zone.a', 'node-exporter.a', 'alertmanager.a'] 2021-02-11T12:32:43.862 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2021-02-11T12:32:43.862 INFO:teuthology.misc:Compressing logs... 2021-02-11T12:32:43.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-11T12:33:02.402 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/misc.py", line 1372, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_14fad31725ce3e5fc27890f2a8ef1f21c838d296/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-02-11T12:33:41.593 INFO:teuthology.kill:No teuthology processes running 2021-02-11T12:33:42.866 INFO:teuthology.kill:Nuking machines: ['smithi091', 'smithi110'] 2021-02-11T12:35:26.447 INFO:teuthology.kill:2021-02-11 12:35:26,447.447 INFO:teuthology.nuke:targets: 2021-02-11T12:35:26.448 INFO:teuthology.kill: smithi091.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXivQZ5qk4nWSchy1vnjW3yJ6rnEYQUVXhOHmoWzz4mMqCzYlXRDKTWgi7InVyLC9YnUXKz2TeeNqkDCJAFfnVTITLHwmgrOsmZnie1rtwr4mnPw7Envxx/+v+YLihqqljx58LdjvgYHtvJpRP4r1jpFOqeuTEyntawzrBScY0MMb82USaRd2CUOV7EI95uGJtDHuATTMAnF2kPiAACXF9pHJ4IbkjU+ZhBos7Ro7LLPDf8/9qZZIte5xRSX5WUW1vIpykRN8SBWfIPrkiieCmp9vudmAmsfTMFgPUrgRXplv8KVOw+hd/PTu2u6ty6jZIOGW9S7USgfWRLmG9IVcHdPBZJWORUS1h3fUcxURO1VzdOIK9aDI06z0ar2PdpUHaKphKL8As+0FmFdrQRaQ5r3h3Rq6zh2IOEU1ggpVqSYJ0IFi/BgOPWYAFJtBv+LxisJsDcx16DH15S466/Nn5EPa6ih45LArr79LIZu3FoIDSCn6ieoqu5MS1bIylkEE= 2021-02-11T12:35:26.448 INFO:teuthology.kill: smithi110.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFxEw4RkOqPutpdvA/M6ZM2A7frXxrH4Ouib8CCIOuMPf0tkZGDxx05Cof3+fV/7rflDqC1xlTH2PiLgI4cgerxP1I5FGMbpr/yTke/bNWhjq+/7Bu+pxuhP6G8i4BpJ/iGYaezVcwrb/EIImcKFFrZ5WdHfuc/0nmSFvBnHm5fnNVm1d1y6WkbeHtugEbfEQt0XIJjQskBiNGUDnFet6EIVOcFnLCG6V9DPtfAZqbUrosHZm/abhet2aaAe/I/0wv1Q8AAQj8w0rrmelcOUl4DgioeVEPHYZ1/twAzR1jOjqVpJ2JYkqr4AAIGOFBiEvJ1UDwio8/SdmN6wsjmO8fr5d7OBW5n3naGKiFuJNBwjac4niGswNFLxgZGwpd0oyqX7ViOP5GrBteBoSzGOEWiHJGo/IasPaOx2vBMLA1Rz1JLESqOzz4hS8XrZdPb/ohgio2p9JGeeqWqeaXiovI7RU+w3S1PA/+GTm1AMXnM/lm8t4xxHNtxthqonTCWkk= 2021-02-11T12:35:26.471 INFO:teuthology.kill:2021-02-11 12:35:26,471.471 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-11T12:35:26.474 INFO:teuthology.kill:2021-02-11 12:35:26,473.473 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-11T12:35:26.532 INFO:teuthology.kill:2021-02-11 12:35:26,532.532 INFO:teuthology.orchestra.console:Power off smithi110 2021-02-11T12:35:26.548 INFO:teuthology.kill:2021-02-11 12:35:26,548.548 INFO:teuthology.orchestra.console:Power off smithi091 2021-02-11T12:35:26.772 INFO:teuthology.kill:2021-02-11 12:35:26,772.772 INFO:teuthology.orchestra.console:Power off for smithi110 completed 2021-02-11T12:37:17.412 INFO:teuthology.kill:2021-02-11 12:35:26,789.789 INFO:teuthology.orchestra.console:Power off for smithi091 completed 2021-02-11T12:37:17.412 INFO:teuthology.kill:2021-02-11 12:35:26,922.922 INFO:teuthology.lock.ops:unlocked smithi110.front.sepia.ceph.com 2021-02-11T12:37:17.412 INFO:teuthology.kill:2021-02-11 12:35:26,930.930 INFO:teuthology.lock.ops:unlocked smithi091.front.sepia.ceph.com 2021-02-11T12:39:17.501 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-02-11T12:39:25.318 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-02-11T12:41:00.438 INFO:teuthology.nuke:Checking targets against current locks 2021-02-11T12:41:00.577 INFO:teuthology.nuke:Not nuking smithi091.front.sepia.ceph.com because description doesn't match 2021-02-11T12:41:00.577 INFO:teuthology.nuke:Not nuking smithi110.front.sepia.ceph.com because description doesn't match