2021-03-04T02:08:47.230 INFO:root:teuthology version: 1.1.0-3082387b 2021-03-04T02:08:47.327 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi179.front.sepia.ceph.com' reimaging 2021-03-04T02:08:47.328 INFO:teuthology.lock.ops:Updating [ubuntu@smithi179.front.sepia.ceph.com]: reset os type and version on server 2021-03-04T02:08:47.329 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2021-03-04T02:08:47.343 INFO:teuthology.lock.ops:Node 'ubuntu@smithi179.front.sepia.ceph.com' reimaging is complete 2021-03-04T02:08:47.343 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging 2021-03-04T02:08:47.344 INFO:teuthology.lock.ops:Updating [ubuntu@smithi185.front.sepia.ceph.com]: reset os type and version on server 2021-03-04T02:08:47.344 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2021-03-04T02:08:47.360 INFO:teuthology.lock.ops:Node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging is complete 2021-03-04T02:08:47.552 INFO:teuthology.provision.fog.smithi179:Scheduling deploy of rhel 8.3 2021-03-04T02:08:47.554 INFO:teuthology.provision.fog.smithi185:Scheduling deploy of rhel 8.3 2021-03-04T02:08:47.911 INFO:teuthology.orchestra.console:Power off smithi179 2021-03-04T02:08:47.945 INFO:teuthology.orchestra.console:Power off smithi185 2021-03-04T02:08:56.279 INFO:teuthology.orchestra.console:Power off for smithi179 completed 2021-03-04T02:08:56.303 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2021-03-04T02:08:56.381 INFO:teuthology.orchestra.console:Power on smithi179 2021-03-04T02:08:56.407 INFO:teuthology.orchestra.console:Power on smithi185 2021-03-04T02:09:04.748 INFO:teuthology.orchestra.console:Power on for smithi179 completed 2021-03-04T02:09:04.767 INFO:teuthology.orchestra.console:Power on for smithi185 completed 2021-03-04T02:09:04.850 INFO:teuthology.provision.fog.smithi179:Waiting for deploy to finish 2021-03-04T02:09:04.868 INFO:teuthology.provision.fog.smithi185:Waiting for deploy to finish 2021-03-04T02:13:43.042 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-04T02:13:43.331 INFO:teuthology.provision.fog.smithi179:Node is ready 2021-03-04T02:13:43.348 INFO:teuthology.orchestra.run.smithi179.stdout:smithi179.front.sepia.ceph.com 2021-03-04T02:13:43.405 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179 smithi179.front.sepia.ceph.com smithi179 2021-03-04T02:13:43.722 INFO:teuthology.provision.fog.smithi179:Deploy complete! 2021-03-04T02:13:58.187 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-04T02:13:58.527 INFO:teuthology.provision.fog.smithi185:Node is ready 2021-03-04T02:13:58.541 INFO:teuthology.orchestra.run.smithi185.stdout:smithi185.front.sepia.ceph.com 2021-03-04T02:13:58.596 INFO:teuthology.orchestra.run.smithi185.stdout:172.21.15.185 smithi185.front.sepia.ceph.com smithi185 2021-03-04T02:13:58.923 INFO:teuthology.provision.fog.smithi185:Deploy complete! 2021-03-04T02:13:59.040 INFO:teuthology.lock.ops:Checking smithi179.front.sepia.ceph.com 2021-03-04T02:13:59.041 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-04T02:13:59.056 INFO:teuthology.lock.ops:Checking smithi185.front.sepia.ceph.com 2021-03-04T02:13:59.056 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-04T02:13:59.069 INFO:teuthology.lock.ops:Updating [smithi179.front.sepia.ceph.com]: set os type and version on server 2021-03-04T02:13:59.070 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-04T02:13:59.359 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2021-03-04T02:13:59.413 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="Red Hat Enterprise Linux" 2021-03-04T02:13:59.413 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="8.3 (Ootpa)" 2021-03-04T02:13:59.413 INFO:teuthology.orchestra.run.smithi179.stdout:ID="rhel" 2021-03-04T02:13:59.413 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="fedora" 2021-03-04T02:13:59.414 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="8.3" 2021-03-04T02:13:59.414 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el8" 2021-03-04T02:13:59.414 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-03-04T02:13:59.414 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2021-03-04T02:13:59.415 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-03-04T02:13:59.415 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://www.redhat.com/" 2021-03-04T02:13:59.415 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-03-04T02:13:59.415 INFO:teuthology.orchestra.run.smithi179.stdout: 2021-03-04T02:13:59.416 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-03-04T02:13:59.416 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-03-04T02:13:59.416 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-03-04T02:13:59.416 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-03-04T02:13:59.417 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2021-03-04T02:13:59.434 INFO:teuthology.lock.ops:Updating [smithi185.front.sepia.ceph.com]: set os type and version on server 2021-03-04T02:13:59.434 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-04T02:13:59.713 INFO:teuthology.orchestra.run.smithi185.stdout:x86_64 2021-03-04T02:13:59.768 INFO:teuthology.orchestra.run.smithi185.stdout:NAME="Red Hat Enterprise Linux" 2021-03-04T02:13:59.768 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION="8.3 (Ootpa)" 2021-03-04T02:13:59.768 INFO:teuthology.orchestra.run.smithi185.stdout:ID="rhel" 2021-03-04T02:13:59.769 INFO:teuthology.orchestra.run.smithi185.stdout:ID_LIKE="fedora" 2021-03-04T02:13:59.769 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_ID="8.3" 2021-03-04T02:13:59.769 INFO:teuthology.orchestra.run.smithi185.stdout:PLATFORM_ID="platform:el8" 2021-03-04T02:13:59.769 INFO:teuthology.orchestra.run.smithi185.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-03-04T02:13:59.769 INFO:teuthology.orchestra.run.smithi185.stdout:ANSI_COLOR="0;31" 2021-03-04T02:13:59.770 INFO:teuthology.orchestra.run.smithi185.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-03-04T02:13:59.770 INFO:teuthology.orchestra.run.smithi185.stdout:HOME_URL="https://www.redhat.com/" 2021-03-04T02:13:59.770 INFO:teuthology.orchestra.run.smithi185.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-03-04T02:13:59.770 INFO:teuthology.orchestra.run.smithi185.stdout: 2021-03-04T02:13:59.771 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-03-04T02:13:59.771 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-03-04T02:13:59.771 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-03-04T02:13:59.771 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-03-04T02:13:59.772 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server 2021-03-04T02:13:59.838 INFO:teuthology.dispatcher.supervisor:Running job 5931143 2021-03-04T02:13:59.838 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/bin/teuthology -v --owner scheduled_jdillaman@cube-1 --archive /home/teuthworker/archive/jdillaman-2021-03-03_14:49:58-rbd-wip-jd-testing-testing-basic-smithi/5931143 --name jdillaman-2021-03-03_14:49:58-rbd-wip-jd-testing-testing-basic-smithi --description rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/rbd_api_tests_copy_on_read} -- /home/teuthworker/archive/jdillaman-2021-03-03_14:49:58-rbd-wip-jd-testing-testing-basic-smithi/5931143/orig.config.yaml 2021-03-04T02:13:59.841 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/jdillaman-2021-03-03_14:49:58-rbd-wip-jd-testing-testing-basic-smithi/5931143 2021-03-04T02:13:59.841 INFO:teuthology.dispatcher.supervisor:Job PID: 7662 2021-03-04T02:13:59.842 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-03-04T14:14:51.479 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-03-04T14:14:51.782 INFO:teuthology.kill:Killing Pids: {7662} 2021-03-04T14:14:53.783 INFO:teuthology.kill:Nuking machines: ['smithi179', 'smithi185'] 2021-03-04T14:14:55.866 INFO:teuthology.kill:2021-03-04 14:14:55,866.866 INFO:teuthology.nuke:targets: 2021-03-04T14:14:55.869 INFO:teuthology.kill: smithi179.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCnrLOYnzTwV0OnA9lZkpgzCGTFHzMF8iGe1pCnTTrUUFYXkXNRSJcjaXM9MdlbO2MaVCyQNV7mnTUZYOwyZatoU29EfBDvWXugCfBKWAOI7WGSTcLoXdWAjFFhrnZ1XUiS3uxQiLlBunstxKdIWLHqdNZHTI9cJcw1E+U7KX6GRQgSK8LUJCG2YdYEnCmI563vX+ZTr5LNJDSlb2EnHniL+VVrQR49XebsLupWbKz5AT4vQ2LuK92LCGwxF+OQ/s50jMUEBtFOi4SBz9esK2N2Nci92Nbx2UfG4yfs2+88ByL+nzs6tTkP77RWnsorGxsSTsOcR7mV11S996KFEJoNLcVSRtvtzq/XkiDlwGv2dbZ+UjmibmwFU5KpxwwT39IvXqyu7lGF1b0wbFXAaNLqH50N7gNdJvsOP9cu56f3TkGCw2z/axBb63DxH/DvQA2c26YELVxthF+AORGrRhqadhuWUyDOkC9EWhv/uklTbQmOZ1WNgizhXjOLvHe3MdM= 2021-03-04T14:14:55.870 INFO:teuthology.kill: smithi185.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqod2IE/yfPEsDbAB3LlcDqbFCpTrj/4LARt3wj3AllJMR1XJutm1bIO/mm7d+caYcw7tLRxBwG2PkTerjSm1uEOB61AGVSFfgn7jJD+X2J1R5KvVKguxc2hteJldqHlQ7AMK86B2T7zsi5pnudiQgJ6IxVh7bTz34gTeemvm336+xC2LyUSBY3ztSv6yXyIB02PDoAwFWeQUIWmLU4XlS81GL/jUDs94ym79w0Mys1fcDIyOC8n3CilauwzzeZRnJPJ2znqNFG/CvTb0AOA+v0B1Aao8RNrGfmXFN7wrvRV6Cb2v41nKPv20FUpUvFoyoW1XXqpRJRKcupfwTedIOCMpOBeRPRE+d83V9gOym/pMugV4wD4ABDIqlLFG220ZByq7x7gM8BCdefyOQgzMReH1PPqBuNR0aEaNeZ5p+JphwCAT9ASB1ZY8OEaO3DIWc2Y4zTf49z4gnuQogYPsyUhOMUBpAr+UtT6/A7jb3y9Mv4uO4GAoRZ1Ezwf+pGoM= 2021-03-04T14:14:55.870 INFO:teuthology.kill:2021-03-04 14:14:55,866.866 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-04T14:14:55.870 INFO:teuthology.kill:2021-03-04 14:14:55,870.870 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-04T14:14:55.943 INFO:teuthology.kill:2021-03-04 14:14:55,943.943 INFO:teuthology.orchestra.console:Power off smithi185 2021-03-04T14:14:55.973 INFO:teuthology.kill:2021-03-04 14:14:55,972.972 INFO:teuthology.orchestra.console:Power off smithi179 2021-03-04T14:15:04.385 INFO:teuthology.kill:2021-03-04 14:15:04,385.385 INFO:teuthology.orchestra.console:Power off for smithi179 completed 2021-03-04T14:15:08.492 INFO:teuthology.kill:2021-03-04 14:15:08,491.491 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2021-03-04T14:15:08.792 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-03-04T14:15:08.796 INFO:teuthology.task.internal:roles: ubuntu@smithi185.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-03-04T14:15:08.797 INFO:teuthology.misc:Compressing logs... 2021-03-04T14:15:08.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-04T14:15:27.171 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/misc.py", line 1374, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-03-04T14:15:27.228 INFO:teuthology.kill:No teuthology processes running 2021-03-04T14:15:29.321 INFO:teuthology.kill:Nuking machines: ['smithi179', 'smithi185'] 2021-03-04T14:15:30.933 INFO:teuthology.kill:2021-03-04 14:15:30,932.932 INFO:teuthology.nuke:targets: 2021-03-04T14:15:30.936 INFO:teuthology.kill: smithi179.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCnrLOYnzTwV0OnA9lZkpgzCGTFHzMF8iGe1pCnTTrUUFYXkXNRSJcjaXM9MdlbO2MaVCyQNV7mnTUZYOwyZatoU29EfBDvWXugCfBKWAOI7WGSTcLoXdWAjFFhrnZ1XUiS3uxQiLlBunstxKdIWLHqdNZHTI9cJcw1E+U7KX6GRQgSK8LUJCG2YdYEnCmI563vX+ZTr5LNJDSlb2EnHniL+VVrQR49XebsLupWbKz5AT4vQ2LuK92LCGwxF+OQ/s50jMUEBtFOi4SBz9esK2N2Nci92Nbx2UfG4yfs2+88ByL+nzs6tTkP77RWnsorGxsSTsOcR7mV11S996KFEJoNLcVSRtvtzq/XkiDlwGv2dbZ+UjmibmwFU5KpxwwT39IvXqyu7lGF1b0wbFXAaNLqH50N7gNdJvsOP9cu56f3TkGCw2z/axBb63DxH/DvQA2c26YELVxthF+AORGrRhqadhuWUyDOkC9EWhv/uklTbQmOZ1WNgizhXjOLvHe3MdM= 2021-03-04T14:15:30.937 INFO:teuthology.kill: smithi185.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqod2IE/yfPEsDbAB3LlcDqbFCpTrj/4LARt3wj3AllJMR1XJutm1bIO/mm7d+caYcw7tLRxBwG2PkTerjSm1uEOB61AGVSFfgn7jJD+X2J1R5KvVKguxc2hteJldqHlQ7AMK86B2T7zsi5pnudiQgJ6IxVh7bTz34gTeemvm336+xC2LyUSBY3ztSv6yXyIB02PDoAwFWeQUIWmLU4XlS81GL/jUDs94ym79w0Mys1fcDIyOC8n3CilauwzzeZRnJPJ2znqNFG/CvTb0AOA+v0B1Aao8RNrGfmXFN7wrvRV6Cb2v41nKPv20FUpUvFoyoW1XXqpRJRKcupfwTedIOCMpOBeRPRE+d83V9gOym/pMugV4wD4ABDIqlLFG220ZByq7x7gM8BCdefyOQgzMReH1PPqBuNR0aEaNeZ5p+JphwCAT9ASB1ZY8OEaO3DIWc2Y4zTf49z4gnuQogYPsyUhOMUBpAr+UtT6/A7jb3y9Mv4uO4GAoRZ1Ezwf+pGoM= 2021-03-04T14:15:30.960 INFO:teuthology.kill:2021-03-04 14:15:30,959.959 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-04T14:15:30.963 INFO:teuthology.kill:2021-03-04 14:15:30,962.962 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-04T14:15:31.024 INFO:teuthology.kill:2021-03-04 14:15:31,024.024 INFO:teuthology.orchestra.console:Power off smithi185 2021-03-04T14:15:31.045 INFO:teuthology.kill:2021-03-04 14:15:31,044.044 INFO:teuthology.orchestra.console:Power off smithi179 2021-03-04T14:15:31.196 INFO:teuthology.kill:2021-03-04 14:15:31,196.196 INFO:teuthology.orchestra.console:Power off for smithi179 completed 2021-03-04T14:15:31.202 INFO:teuthology.kill:2021-03-04 14:15:31,202.202 INFO:teuthology.orchestra.console:Power off for smithi185 completed 2021-03-04T14:15:31.349 INFO:teuthology.kill:2021-03-04 14:15:31,348.348 INFO:teuthology.lock.ops:unlocked smithi179.front.sepia.ceph.com 2021-03-04T14:15:31.352 INFO:teuthology.kill:2021-03-04 14:15:31,352.352 INFO:teuthology.lock.ops:unlocked smithi185.front.sepia.ceph.com 2021-03-04T14:17:31.643 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-03-04T14:17:31.732 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-03-04T14:17:31.733 INFO:teuthology.nuke:Checking targets against current locks 2021-03-04T14:17:31.809 INFO:teuthology.nuke:Not nuking smithi179.front.sepia.ceph.com because description doesn't match 2021-03-04T14:17:31.809 INFO:teuthology.nuke:Not nuking smithi185.front.sepia.ceph.com because description doesn't match