2021-06-01T06:07:14.323 INFO:root:teuthology version: 1.1.0-f359b10d 2021-06-01T06:07:14.433 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi106.front.sepia.ceph.com' reimaging 2021-06-01T06:07:14.434 INFO:teuthology.lock.ops:Updating [ubuntu@smithi106.front.sepia.ceph.com]: reset os type and version on server 2021-06-01T06:07:14.434 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2021-06-01T06:07:14.453 INFO:teuthology.lock.ops:Node 'ubuntu@smithi106.front.sepia.ceph.com' reimaging is complete 2021-06-01T06:07:14.453 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi164.front.sepia.ceph.com' reimaging 2021-06-01T06:07:14.453 INFO:teuthology.lock.ops:Updating [ubuntu@smithi164.front.sepia.ceph.com]: reset os type and version on server 2021-06-01T06:07:14.454 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2021-06-01T06:07:14.475 INFO:teuthology.lock.ops:Node 'ubuntu@smithi164.front.sepia.ceph.com' reimaging is complete 2021-06-01T06:07:14.665 INFO:teuthology.provision.fog.smithi106:Scheduling deploy of centos 8.2 2021-06-01T06:07:14.702 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of centos 8.2 2021-06-01T06:07:14.995 INFO:teuthology.orchestra.console:Power off smithi106 2021-06-01T06:07:15.036 INFO:teuthology.orchestra.console:Power off smithi164 2021-06-01T06:07:27.486 INFO:teuthology.orchestra.console:Power off for smithi106 completed 2021-06-01T06:07:27.516 INFO:teuthology.orchestra.console:Power off for smithi164 completed 2021-06-01T06:07:27.589 INFO:teuthology.orchestra.console:Power on smithi106 2021-06-01T06:07:27.620 INFO:teuthology.orchestra.console:Power on smithi164 2021-06-01T06:07:35.960 INFO:teuthology.orchestra.console:Power on for smithi106 completed 2021-06-01T06:07:36.062 INFO:teuthology.provision.fog.smithi106:Waiting for deploy to finish 2021-06-01T06:07:36.078 INFO:teuthology.orchestra.console:Power on for smithi164 completed 2021-06-01T06:07:36.178 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish 2021-06-01T06:12:08.011 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-01T06:12:08.310 INFO:teuthology.provision.fog.smithi106:Node is ready 2021-06-01T06:12:08.324 INFO:teuthology.orchestra.run.smithi106.stdout:smithi106.front.sepia.ceph.com 2021-06-01T06:12:08.378 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106 smithi106.front.sepia.ceph.com smithi106 2021-06-01T06:12:08.738 INFO:teuthology.provision.fog.smithi106:Deploy complete! 2021-06-01T06:12:23.062 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-01T06:12:23.376 INFO:teuthology.provision.fog.smithi164:Node is ready 2021-06-01T06:12:23.390 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164.front.sepia.ceph.com 2021-06-01T06:12:23.445 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164 2021-06-01T06:12:23.765 INFO:teuthology.provision.fog.smithi164:Deploy complete! 2021-06-01T06:12:23.880 INFO:teuthology.lock.ops:Checking smithi106.front.sepia.ceph.com 2021-06-01T06:12:23.880 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-01T06:12:23.896 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com 2021-06-01T06:12:23.896 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-01T06:12:23.909 INFO:teuthology.lock.ops:Updating [smithi106.front.sepia.ceph.com]: set os type and version on server 2021-06-01T06:12:23.910 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-01T06:12:24.184 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2021-06-01T06:12:24.239 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Linux" 2021-06-01T06:12:24.240 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="8 (Core)" 2021-06-01T06:12:24.240 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos" 2021-06-01T06:12:24.240 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora" 2021-06-01T06:12:24.241 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="8" 2021-06-01T06:12:24.241 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el8" 2021-06-01T06:12:24.241 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-06-01T06:12:24.241 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2021-06-01T06:12:24.242 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-06-01T06:12:24.242 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://www.centos.org/" 2021-06-01T06:12:24.242 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-06-01T06:12:24.242 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-06-01T06:12:24.242 INFO:teuthology.orchestra.run.smithi106.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-06-01T06:12:24.243 INFO:teuthology.orchestra.run.smithi106.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-06-01T06:12:24.243 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-06-01T06:12:24.243 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-06-01T06:12:24.243 INFO:teuthology.orchestra.run.smithi106.stdout: 2021-06-01T06:12:24.245 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2021-06-01T06:12:24.267 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server 2021-06-01T06:12:24.267 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-01T06:12:24.530 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2021-06-01T06:12:24.585 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Linux" 2021-06-01T06:12:24.586 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8 (Core)" 2021-06-01T06:12:24.586 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2021-06-01T06:12:24.586 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2021-06-01T06:12:24.586 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8" 2021-06-01T06:12:24.587 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8" 2021-06-01T06:12:24.587 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-06-01T06:12:24.591 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2021-06-01T06:12:24.591 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-06-01T06:12:24.591 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.centos.org/" 2021-06-01T06:12:24.591 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-06-01T06:12:24.592 INFO:teuthology.orchestra.run.smithi164.stdout: 2021-06-01T06:12:24.592 INFO:teuthology.orchestra.run.smithi164.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-06-01T06:12:24.592 INFO:teuthology.orchestra.run.smithi164.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-06-01T06:12:24.592 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-06-01T06:12:24.592 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-06-01T06:12:24.593 INFO:teuthology.orchestra.run.smithi164.stdout: 2021-06-01T06:12:24.593 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2021-06-01T06:12:24.730 INFO:teuthology.dispatcher.supervisor:Running job 6144591 2021-06-01T06:12:24.730 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-06-01_04:17:03-fs-pacific-distro-basic-smithi/6144591 --name teuthology-2021-06-01_04:17:03-fs-pacific-distro-basic-smithi --description fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} -- /home/teuthworker/archive/teuthology-2021-06-01_04:17:03-fs-pacific-distro-basic-smithi/6144591/orig.config.yaml 2021-06-01T06:12:24.733 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-06-01_04:17:03-fs-pacific-distro-basic-smithi/6144591 2021-06-01T06:12:24.734 INFO:teuthology.dispatcher.supervisor:Job PID: 26495 2021-06-01T06:12:24.735 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-06-01T18:13:13.990 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-06-01T18:13:14.234 INFO:teuthology.kill:Killing Pids: {26495} 2021-06-01T18:13:15.351 INFO:teuthology.kill:Nuking machines: ['smithi106', 'smithi164'] 2021-06-01T18:13:16.244 INFO:teuthology.kill:2021-06-01 18:13:16,244.244 INFO:teuthology.nuke:targets: 2021-06-01T18:13:16.245 INFO:teuthology.kill: smithi106.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLByAGlNhuiSF2F+Uy532Ic3QA6p62F2dz2QqtefmtoGVYjNAHFfdwbmu/UQBoY9C/nGRi2oF5ewvTK5yPIx6BuKESIgygNY+qQrb5bpYWNYYSuEs0Kti4Qnr5/C+xjRD+AgLGhBBNutoYUOyd0JZNPvY5aaHlftzckBhKDz6Jvh9NO+wR39pn3oSuoKAYA8X39PEx7B4cH1uqQjXtFJcZ8AQvv/gi/1+I+gUhXP5/3lQiJ+BXBsdVIP7jqXn7waRDfxLJrPOgSdCAMxkGmgTouON0NBqxtynnNnK46tm+CqHuNW5WaW3Fff5vIdgwqq0OT+e/1LvNd0Lgw1OknZo+bBfAdN8ZE+wwUhTek5wQsE7smtPHaK80rBk/tg+YxGvNKS4LJcrW0kw9raZ+m9IwPZcAaa/nBg6LY+Fyb7y6lrkAS5+OnPgF+4S4990l6dGQunkMvPVv5RD90F9mrzLiuESv5vaguY1Jp33B3hxN4yZ6UxZW14xaCfujBo2GJms= 2021-06-01T18:13:16.245 INFO:teuthology.kill: smithi164.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUUoIqjak0uVOrSS3vPxyN7m6GeRQcG2rTuU6rDMOXCiUfmJJZZKdTfHJ7PZ+KleZYCHHZCyjvGKCH8o7q3fi2EkSTmEFZ/tLxtqdGo3DVH4Ci3sxoMfoU97+nWtsTgpUj2antvbqNklG73yuX+7vgI9KHzqjIKx5AUrzEbJV1PJ4OHFDrtRSzBkKZ/CgkyNf7evuykBDg5i6vVbOVltM9NRyPvJZ1m9HfVubhwfYVzcBzoG1nBPqnMX723RKusMBlZ0hIvV6CLEMYmbSn/EIWNBp92T5ul7XhfQA4RBs43MjIFsKv9Hraji5pBhH3GmHUyjJroZaCoOsCK5s2GZurhx+T0hWerVCy6QAR4ur3dORqcOyWRnAW9YnHHuaLw5R1ZKd5aITiX8de/E77u22mmmme6dtqyZw1e8a3W4e2yxbHGDsyDZoJK+pg7vCRhl0fTpabaR0Pp/EbgNQOWDhvKbmr4S4k8BDgyImtyTMdRD4hmNI2W5mbYeDIq5PFsS0= 2021-06-01T18:13:16.245 INFO:teuthology.kill:2021-06-01 18:13:16,244.244 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-01T18:13:16.246 INFO:teuthology.kill:2021-06-01 18:13:16,246.246 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-01T18:13:16.303 INFO:teuthology.kill:2021-06-01 18:13:16,303.303 INFO:teuthology.orchestra.console:Power off smithi106 2021-06-01T18:13:16.316 INFO:teuthology.kill:2021-06-01 18:13:16,315.315 INFO:teuthology.orchestra.console:Power off smithi164 2021-06-01T18:13:24.670 INFO:teuthology.kill:2021-06-01 18:13:24,670.670 INFO:teuthology.orchestra.console:Power off for smithi106 completed 2021-06-01T18:13:24.674 INFO:teuthology.kill:2021-06-01 18:13:24,674.674 INFO:teuthology.orchestra.console:Power off for smithi164 completed 2021-06-01T18:13:24.887 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2021-06-01T18:13:24.888 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2021-06-01T18:13:24.888 INFO:teuthology.misc:Compressing logs... 2021-06-01T18:13:24.889 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-01T18:14:24.952 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-06-01T18:14:25.026 INFO:teuthology.kill:No teuthology processes running 2021-06-01T18:14:26.036 INFO:teuthology.kill:Nuking machines: ['smithi106', 'smithi164'] 2021-06-01T18:14:26.916 INFO:teuthology.kill:2021-06-01 18:14:26,916.916 INFO:teuthology.nuke:targets: 2021-06-01T18:14:26.917 INFO:teuthology.kill: smithi106.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLByAGlNhuiSF2F+Uy532Ic3QA6p62F2dz2QqtefmtoGVYjNAHFfdwbmu/UQBoY9C/nGRi2oF5ewvTK5yPIx6BuKESIgygNY+qQrb5bpYWNYYSuEs0Kti4Qnr5/C+xjRD+AgLGhBBNutoYUOyd0JZNPvY5aaHlftzckBhKDz6Jvh9NO+wR39pn3oSuoKAYA8X39PEx7B4cH1uqQjXtFJcZ8AQvv/gi/1+I+gUhXP5/3lQiJ+BXBsdVIP7jqXn7waRDfxLJrPOgSdCAMxkGmgTouON0NBqxtynnNnK46tm+CqHuNW5WaW3Fff5vIdgwqq0OT+e/1LvNd0Lgw1OknZo+bBfAdN8ZE+wwUhTek5wQsE7smtPHaK80rBk/tg+YxGvNKS4LJcrW0kw9raZ+m9IwPZcAaa/nBg6LY+Fyb7y6lrkAS5+OnPgF+4S4990l6dGQunkMvPVv5RD90F9mrzLiuESv5vaguY1Jp33B3hxN4yZ6UxZW14xaCfujBo2GJms= 2021-06-01T18:14:26.918 INFO:teuthology.kill: smithi164.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUUoIqjak0uVOrSS3vPxyN7m6GeRQcG2rTuU6rDMOXCiUfmJJZZKdTfHJ7PZ+KleZYCHHZCyjvGKCH8o7q3fi2EkSTmEFZ/tLxtqdGo3DVH4Ci3sxoMfoU97+nWtsTgpUj2antvbqNklG73yuX+7vgI9KHzqjIKx5AUrzEbJV1PJ4OHFDrtRSzBkKZ/CgkyNf7evuykBDg5i6vVbOVltM9NRyPvJZ1m9HfVubhwfYVzcBzoG1nBPqnMX723RKusMBlZ0hIvV6CLEMYmbSn/EIWNBp92T5ul7XhfQA4RBs43MjIFsKv9Hraji5pBhH3GmHUyjJroZaCoOsCK5s2GZurhx+T0hWerVCy6QAR4ur3dORqcOyWRnAW9YnHHuaLw5R1ZKd5aITiX8de/E77u22mmmme6dtqyZw1e8a3W4e2yxbHGDsyDZoJK+pg7vCRhl0fTpabaR0Pp/EbgNQOWDhvKbmr4S4k8BDgyImtyTMdRD4hmNI2W5mbYeDIq5PFsS0= 2021-06-01T18:14:26.937 INFO:teuthology.kill:2021-06-01 18:14:26,937.937 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-01T18:14:26.940 INFO:teuthology.kill:2021-06-01 18:14:26,940.940 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-01T18:14:26.993 INFO:teuthology.kill:2021-06-01 18:14:26,993.993 INFO:teuthology.orchestra.console:Power off smithi164 2021-06-01T18:14:27.005 INFO:teuthology.kill:2021-06-01 18:14:27,005.005 INFO:teuthology.orchestra.console:Power off smithi106 2021-06-01T18:14:27.136 INFO:teuthology.kill:2021-06-01 18:14:27,136.136 INFO:teuthology.orchestra.console:Power off for smithi164 completed 2021-06-01T18:14:27.150 INFO:teuthology.kill:2021-06-01 18:14:27,149.149 INFO:teuthology.orchestra.console:Power off for smithi106 completed 2021-06-01T18:14:27.276 INFO:teuthology.kill:2021-06-01 18:14:27,275.275 INFO:teuthology.lock.ops:unlocked smithi164.front.sepia.ceph.com 2021-06-01T18:14:27.284 INFO:teuthology.kill:2021-06-01 18:14:27,284.284 INFO:teuthology.lock.ops:unlocked smithi106.front.sepia.ceph.com 2021-06-01T18:16:27.566 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-06-01T18:16:27.659 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-06-01T18:16:27.660 INFO:teuthology.nuke:Checking targets against current locks 2021-06-01T18:16:27.724 INFO:teuthology.nuke:Not nuking smithi164.front.sepia.ceph.com because description doesn't match 2021-06-01T18:16:27.725 INFO:teuthology.nuke:Not nuking smithi106.front.sepia.ceph.com because description doesn't match