2021-02-17T06:28:08.955 INFO:root:teuthology version: 1.1.0-071a4df2 2021-02-17T06:28:09.229 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi038.front.sepia.ceph.com' reimaging 2021-02-17T06:28:09.233 INFO:teuthology.lock.ops:Updating [ubuntu@smithi038.front.sepia.ceph.com]: reset os type and version on server 2021-02-17T06:28:09.233 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2021-02-17T06:28:09.260 INFO:teuthology.lock.ops:Node 'ubuntu@smithi038.front.sepia.ceph.com' reimaging is complete 2021-02-17T06:28:09.260 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi074.front.sepia.ceph.com' reimaging 2021-02-17T06:28:09.261 INFO:teuthology.lock.ops:Updating [ubuntu@smithi074.front.sepia.ceph.com]: reset os type and version on server 2021-02-17T06:28:09.261 INFO:teuthology.lock.ops:Updating smithi074.front.sepia.ceph.com on lock server 2021-02-17T06:28:09.280 INFO:teuthology.lock.ops:Node 'ubuntu@smithi074.front.sepia.ceph.com' reimaging is complete 2021-02-17T06:28:09.466 INFO:teuthology.provision.fog.smithi038:Scheduling deploy of centos 8.2 2021-02-17T06:28:09.487 INFO:teuthology.provision.fog.smithi074:Scheduling deploy of centos 8.2 2021-02-17T06:28:09.929 INFO:teuthology.orchestra.console:Power off smithi038 2021-02-17T06:28:10.043 INFO:teuthology.orchestra.console:Power off smithi074 2021-02-17T06:28:18.528 INFO:teuthology.orchestra.console:Power off for smithi038 completed 2021-02-17T06:28:18.588 INFO:teuthology.orchestra.console:Power off for smithi074 completed 2021-02-17T06:28:18.632 INFO:teuthology.orchestra.console:Power on smithi038 2021-02-17T06:28:18.716 INFO:teuthology.orchestra.console:Power on smithi074 2021-02-17T06:28:23.004 INFO:teuthology.orchestra.console:Power on for smithi038 completed 2021-02-17T06:28:23.056 INFO:teuthology.orchestra.console:Power on for smithi074 completed 2021-02-17T06:28:23.106 INFO:teuthology.provision.fog.smithi038:Waiting for deploy to finish 2021-02-17T06:28:23.158 INFO:teuthology.provision.fog.smithi074:Waiting for deploy to finish 2021-02-17T06:32:17.628 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-17T06:32:27.334 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-02-17T06:32:28.061 INFO:teuthology.provision.fog.smithi038:Node is ready 2021-02-17T06:32:28.082 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038.front.sepia.ceph.com 2021-02-17T06:32:28.145 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38 smithi038.front.sepia.ceph.com smithi038 2021-02-17T06:32:28.498 INFO:teuthology.provision.fog.smithi038:Deploy complete! 2021-02-17T06:32:29.843 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-17T06:32:50.031 INFO:teuthology.provision.fog.smithi074:Node is ready 2021-02-17T06:32:50.055 INFO:teuthology.orchestra.run.smithi074.stdout:smithi074.front.sepia.ceph.com 2021-02-17T06:32:50.117 INFO:teuthology.orchestra.run.smithi074.stdout:172.21.15.74 smithi074.front.sepia.ceph.com smithi074 2021-02-17T06:32:50.471 INFO:teuthology.provision.fog.smithi074:Deploy complete! 2021-02-17T06:32:50.781 INFO:teuthology.lock.ops:Checking smithi038.front.sepia.ceph.com 2021-02-17T06:32:50.831 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-17T06:32:50.889 INFO:teuthology.lock.ops:Checking smithi074.front.sepia.ceph.com 2021-02-17T06:32:50.895 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-17T06:32:50.917 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: set os type and version on server 2021-02-17T06:32:50.918 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-17T06:32:51.213 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2021-02-17T06:32:51.268 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Linux" 2021-02-17T06:32:51.269 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="8 (Core)" 2021-02-17T06:32:51.269 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos" 2021-02-17T06:32:51.269 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora" 2021-02-17T06:32:51.269 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="8" 2021-02-17T06:32:51.270 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el8" 2021-02-17T06:32:51.270 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-02-17T06:32:51.270 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2021-02-17T06:32:51.271 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-02-17T06:32:51.271 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://www.centos.org/" 2021-02-17T06:32:51.271 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-02-17T06:32:51.271 INFO:teuthology.orchestra.run.smithi038.stdout: 2021-02-17T06:32:51.271 INFO:teuthology.orchestra.run.smithi038.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-02-17T06:32:51.272 INFO:teuthology.orchestra.run.smithi038.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-02-17T06:32:51.272 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-02-17T06:32:51.272 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-02-17T06:32:51.272 INFO:teuthology.orchestra.run.smithi038.stdout: 2021-02-17T06:32:51.274 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2021-02-17T06:32:51.304 INFO:teuthology.lock.ops:Updating [smithi074.front.sepia.ceph.com]: set os type and version on server 2021-02-17T06:32:51.305 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-17T06:32:51.585 INFO:teuthology.orchestra.run.smithi074.stdout:x86_64 2021-02-17T06:32:51.642 INFO:teuthology.orchestra.run.smithi074.stdout:NAME="CentOS Linux" 2021-02-17T06:32:51.642 INFO:teuthology.orchestra.run.smithi074.stdout:VERSION="8 (Core)" 2021-02-17T06:32:51.642 INFO:teuthology.orchestra.run.smithi074.stdout:ID="centos" 2021-02-17T06:32:51.643 INFO:teuthology.orchestra.run.smithi074.stdout:ID_LIKE="rhel fedora" 2021-02-17T06:32:51.643 INFO:teuthology.orchestra.run.smithi074.stdout:VERSION_ID="8" 2021-02-17T06:32:51.643 INFO:teuthology.orchestra.run.smithi074.stdout:PLATFORM_ID="platform:el8" 2021-02-17T06:32:51.643 INFO:teuthology.orchestra.run.smithi074.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-02-17T06:32:51.644 INFO:teuthology.orchestra.run.smithi074.stdout:ANSI_COLOR="0;31" 2021-02-17T06:32:51.644 INFO:teuthology.orchestra.run.smithi074.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-02-17T06:32:51.644 INFO:teuthology.orchestra.run.smithi074.stdout:HOME_URL="https://www.centos.org/" 2021-02-17T06:32:51.644 INFO:teuthology.orchestra.run.smithi074.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-02-17T06:32:51.644 INFO:teuthology.orchestra.run.smithi074.stdout: 2021-02-17T06:32:51.645 INFO:teuthology.orchestra.run.smithi074.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-02-17T06:32:51.645 INFO:teuthology.orchestra.run.smithi074.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-02-17T06:32:51.645 INFO:teuthology.orchestra.run.smithi074.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-02-17T06:32:51.645 INFO:teuthology.orchestra.run.smithi074.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-02-17T06:32:51.645 INFO:teuthology.orchestra.run.smithi074.stdout: 2021-02-17T06:32:51.648 INFO:teuthology.lock.ops:Updating smithi074.front.sepia.ceph.com on lock server 2021-02-17T06:32:51.773 INFO:teuthology.dispatcher.supervisor:Running job 5889717 2021-02-17T06:32:51.774 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/virtualenv/bin/teuthology -v --owner vshankar --archive /home/teuthworker/archive/vshankar-2021-02-17_06:25:42-fs-wip-vshankar-testing-20210216-234922-distro-basic-smithi/5889717 --name vshankar-2021-02-17_06:25:42-fs-wip-vshankar-testing-20210216-234922-distro-basic-smithi --description fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} -- /home/teuthworker/archive/vshankar-2021-02-17_06:25:42-fs-wip-vshankar-testing-20210216-234922-distro-basic-smithi/5889717/orig.config.yaml 2021-02-17T06:32:51.820 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2021-02-17_06:25:42-fs-wip-vshankar-testing-20210216-234922-distro-basic-smithi/5889717 2021-02-17T06:32:51.822 INFO:teuthology.dispatcher.supervisor:Job PID: 13848 2021-02-17T06:32:51.822 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-02-17T18:33:07.996 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-02-17T18:33:08.264 INFO:teuthology.kill:Killing Pids: {13848} 2021-02-17T18:33:09.936 INFO:teuthology.kill:Nuking machines: ['smithi038', 'smithi074'] 2021-02-17T18:33:10.920 INFO:teuthology.kill:2021-02-17 18:33:10,920.920 INFO:teuthology.nuke:targets: 2021-02-17T18:33:10.921 INFO:teuthology.kill: smithi038.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqM3JA5EemR55Gw9CuFFxWzjsarbSxXM2+ALiFJsoE275r+g21zl1GWP0IVydxjmL0SKho1lGRToIlY6z3gt13Ly0KDXOOAweZKck4PxglIEf8yFWC4oUYEtOhIGoE4x8jcTTrarOOwCuEzU/4V5y5bYYujEKtIqP+vNBr5HiSn1nN2L+dLvNDleUkk8L6seDv7uHBH6KmtDdKEGXMkRdX3k47uqAr7XFUBcjlPerYztvewG19m4oANbVKw6V5lpI6engoJjC8BrC16iE0nTyHqSP82E+nhOTFyeV8c7EHl7hlI1bbAiK+TB2jDwaFlw9Gi3C8jNNoOmyVhUoS3b5TdaclZaSa3jsi0hvczZhpRNplVLUcQPKCUOXliCqRPGG5mpdTQy2v6VQ9IkJKa58tPdIDGQv1mu/N6f07wr8G2skQknRjB9TdwSMtp27FXYvx+564+Iaoehk2YyY7YfotDjJCa7wriBU6GtPqvVpfNM5CQZhSJOdspkQSaZoloPs= 2021-02-17T18:33:10.921 INFO:teuthology.kill: smithi074.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDiGoKsDcJvTRkJwkb1K7YzS5iLieibVfA/K7scj+H3LWFlm2qig92dsGO+YG04nQOYxuuNu4+fhMO3zLY05fwEC2RlwArnDdmsv+h8KVF0r+DPzNT4PMibDNsGPBxVNc8yD/pux3e8BnzArZvw1Tj00YJeLIL2LFbcmGjySKmJsCGrllhiw5VA/qNehdZWvakASlVVUoG+30I4aGw1qkrbQCVqWx46KhPhQYO2T7ojL8d8Wri+GhNwaLyk28JPcbh/hfY0eTUghB4PHdtJNhTOFiSzLGskhJ6+p6SBzEfQ8PbGxfxvhEGj1htsplawH2JN4D1YCQNOVS4n4YPeRysLvFTzlA4bcPDSWpeA2mdyGxLYNN0/VgFYHhGYmARBzR7ShzNDnzKGN50P6WSW2tPHF/mE9x8lhNrwxylaT7w2jnpkEloKiLEEa61fhO4afUWxcpkb/U5cz3rE0P/zaTtMK7Foi2/SQj20SyckKVipR/pAzjkGBxGOWNhR4SF2Ni8= 2021-02-17T18:33:10.922 INFO:teuthology.kill:2021-02-17 18:33:10,920.920 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-17T18:33:10.938 INFO:teuthology.kill:2021-02-17 18:33:10,937.937 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-17T18:33:11.005 INFO:teuthology.kill:2021-02-17 18:33:11,005.005 INFO:teuthology.orchestra.console:Power off smithi074 2021-02-17T18:33:11.024 INFO:teuthology.kill:2021-02-17 18:33:11,024.024 INFO:teuthology.orchestra.console:Power off smithi038 2021-02-17T18:33:19.408 INFO:teuthology.kill:2021-02-17 18:33:19,408.408 INFO:teuthology.orchestra.console:Power off for smithi074 completed 2021-02-17T18:33:23.527 INFO:teuthology.kill:2021-02-17 18:33:23,527.527 INFO:teuthology.orchestra.console:Power off for smithi038 completed 2021-02-17T18:33:23.764 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2021-02-17T18:33:23.764 INFO:teuthology.task.internal:roles: ubuntu@smithi074.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2021-02-17T18:33:23.765 INFO:teuthology.misc:Compressing logs... 2021-02-17T18:33:23.765 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-17T18:34:23.771 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/teuthology/misc.py", line 1372, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-02-17T18:34:23.858 INFO:teuthology.kill:No teuthology processes running 2021-02-17T18:34:25.688 INFO:teuthology.kill:Nuking machines: ['smithi038', 'smithi074'] 2021-02-17T18:34:27.162 INFO:teuthology.kill:2021-02-17 18:34:27,162.162 INFO:teuthology.nuke:targets: 2021-02-17T18:34:27.168 INFO:teuthology.kill: smithi038.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqM3JA5EemR55Gw9CuFFxWzjsarbSxXM2+ALiFJsoE275r+g21zl1GWP0IVydxjmL0SKho1lGRToIlY6z3gt13Ly0KDXOOAweZKck4PxglIEf8yFWC4oUYEtOhIGoE4x8jcTTrarOOwCuEzU/4V5y5bYYujEKtIqP+vNBr5HiSn1nN2L+dLvNDleUkk8L6seDv7uHBH6KmtDdKEGXMkRdX3k47uqAr7XFUBcjlPerYztvewG19m4oANbVKw6V5lpI6engoJjC8BrC16iE0nTyHqSP82E+nhOTFyeV8c7EHl7hlI1bbAiK+TB2jDwaFlw9Gi3C8jNNoOmyVhUoS3b5TdaclZaSa3jsi0hvczZhpRNplVLUcQPKCUOXliCqRPGG5mpdTQy2v6VQ9IkJKa58tPdIDGQv1mu/N6f07wr8G2skQknRjB9TdwSMtp27FXYvx+564+Iaoehk2YyY7YfotDjJCa7wriBU6GtPqvVpfNM5CQZhSJOdspkQSaZoloPs= 2021-02-17T18:34:27.168 INFO:teuthology.kill: smithi074.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDiGoKsDcJvTRkJwkb1K7YzS5iLieibVfA/K7scj+H3LWFlm2qig92dsGO+YG04nQOYxuuNu4+fhMO3zLY05fwEC2RlwArnDdmsv+h8KVF0r+DPzNT4PMibDNsGPBxVNc8yD/pux3e8BnzArZvw1Tj00YJeLIL2LFbcmGjySKmJsCGrllhiw5VA/qNehdZWvakASlVVUoG+30I4aGw1qkrbQCVqWx46KhPhQYO2T7ojL8d8Wri+GhNwaLyk28JPcbh/hfY0eTUghB4PHdtJNhTOFiSzLGskhJ6+p6SBzEfQ8PbGxfxvhEGj1htsplawH2JN4D1YCQNOVS4n4YPeRysLvFTzlA4bcPDSWpeA2mdyGxLYNN0/VgFYHhGYmARBzR7ShzNDnzKGN50P6WSW2tPHF/mE9x8lhNrwxylaT7w2jnpkEloKiLEEa61fhO4afUWxcpkb/U5cz3rE0P/zaTtMK7Foi2/SQj20SyckKVipR/pAzjkGBxGOWNhR4SF2Ni8= 2021-02-17T18:34:27.188 INFO:teuthology.kill:2021-02-17 18:34:27,188.188 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-17T18:34:27.191 INFO:teuthology.kill:2021-02-17 18:34:27,191.191 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-17T18:34:27.256 INFO:teuthology.kill:2021-02-17 18:34:27,256.256 INFO:teuthology.orchestra.console:Power off smithi074 2021-02-17T18:34:27.271 INFO:teuthology.kill:2021-02-17 18:34:27,270.270 INFO:teuthology.orchestra.console:Power off smithi038 2021-02-17T18:34:27.417 INFO:teuthology.kill:2021-02-17 18:34:27,417.417 INFO:teuthology.orchestra.console:Power off for smithi074 completed 2021-02-17T18:34:27.429 INFO:teuthology.kill:2021-02-17 18:34:27,429.429 INFO:teuthology.orchestra.console:Power off for smithi038 completed 2021-02-17T18:34:27.564 INFO:teuthology.kill:2021-02-17 18:34:27,563.563 INFO:teuthology.lock.ops:unlocked smithi074.front.sepia.ceph.com 2021-02-17T18:34:27.569 INFO:teuthology.kill:2021-02-17 18:34:27,568.568 INFO:teuthology.lock.ops:unlocked smithi038.front.sepia.ceph.com 2021-02-17T18:36:27.760 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-02-17T18:36:27.855 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-02-17T18:36:27.860 INFO:teuthology.nuke:Checking targets against current locks 2021-02-17T18:36:27.934 INFO:teuthology.nuke:Not nuking smithi074.front.sepia.ceph.com because description doesn't match 2021-02-17T18:36:27.935 INFO:teuthology.nuke:Not nuking smithi038.front.sepia.ceph.com because description doesn't match