2021-05-25T04:18:21.940 INFO:root:teuthology version: 1.1.0-d92ec140 2021-05-25T04:18:22.043 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi112.front.sepia.ceph.com' reimaging 2021-05-25T04:18:22.045 INFO:teuthology.lock.ops:Updating [ubuntu@smithi112.front.sepia.ceph.com]: reset os type and version on server 2021-05-25T04:18:22.046 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2021-05-25T04:18:22.061 INFO:teuthology.lock.ops:Node 'ubuntu@smithi112.front.sepia.ceph.com' reimaging is complete 2021-05-25T04:18:22.062 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi132.front.sepia.ceph.com' reimaging 2021-05-25T04:18:22.062 INFO:teuthology.lock.ops:Updating [ubuntu@smithi132.front.sepia.ceph.com]: reset os type and version on server 2021-05-25T04:18:22.062 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2021-05-25T04:18:22.079 INFO:teuthology.lock.ops:Node 'ubuntu@smithi132.front.sepia.ceph.com' reimaging is complete 2021-05-25T04:18:22.257 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of centos 8.2 2021-05-25T04:18:22.338 INFO:teuthology.provision.fog.smithi132:Scheduling deploy of centos 8.2 2021-05-25T04:18:22.700 INFO:teuthology.orchestra.console:Power off smithi112 2021-05-25T04:18:22.755 INFO:teuthology.orchestra.console:Power off smithi132 2021-05-25T04:18:31.074 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2021-05-25T04:18:31.175 INFO:teuthology.orchestra.console:Power on smithi112 2021-05-25T04:18:35.231 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2021-05-25T04:18:35.333 INFO:teuthology.orchestra.console:Power on smithi132 2021-05-25T04:18:39.555 INFO:teuthology.orchestra.console:Power on for smithi112 completed 2021-05-25T04:18:39.583 INFO:teuthology.orchestra.console:Power on for smithi132 completed 2021-05-25T04:18:39.657 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish 2021-05-25T04:18:39.685 INFO:teuthology.provision.fog.smithi132:Waiting for deploy to finish 2021-05-25T04:23:13.526 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-05-25T04:23:13.811 INFO:teuthology.provision.fog.smithi132:Node is ready 2021-05-25T04:23:13.825 INFO:teuthology.orchestra.run.smithi132.stdout:smithi132.front.sepia.ceph.com 2021-05-25T04:23:13.879 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132 smithi132.front.sepia.ceph.com smithi132 2021-05-25T04:23:14.209 INFO:teuthology.provision.fog.smithi132:Deploy complete! 2021-05-25T04:24:30.348 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-05-25T04:24:50.744 INFO:teuthology.provision.fog.smithi112:Node is ready 2021-05-25T04:24:50.763 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112.front.sepia.ceph.com 2021-05-25T04:24:50.819 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112 smithi112.front.sepia.ceph.com smithi112 2021-05-25T04:24:51.174 INFO:teuthology.provision.fog.smithi112:Deploy complete! 2021-05-25T04:24:51.286 INFO:teuthology.lock.ops:Checking smithi112.front.sepia.ceph.com 2021-05-25T04:24:51.287 INFO:teuthology.lock.ops:New key found. Updating... 2021-05-25T04:24:51.300 INFO:teuthology.lock.ops:Checking smithi132.front.sepia.ceph.com 2021-05-25T04:24:51.300 INFO:teuthology.lock.ops:New key found. Updating... 2021-05-25T04:24:51.313 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: set os type and version on server 2021-05-25T04:24:51.313 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-25T04:24:51.574 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2021-05-25T04:24:51.627 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Linux" 2021-05-25T04:24:51.628 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="8 (Core)" 2021-05-25T04:24:51.628 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2021-05-25T04:24:51.628 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2021-05-25T04:24:51.628 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="8" 2021-05-25T04:24:51.629 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el8" 2021-05-25T04:24:51.629 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-05-25T04:24:51.629 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2021-05-25T04:24:51.629 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-05-25T04:24:51.629 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://www.centos.org/" 2021-05-25T04:24:51.630 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-05-25T04:24:51.630 INFO:teuthology.orchestra.run.smithi112.stdout: 2021-05-25T04:24:51.630 INFO:teuthology.orchestra.run.smithi112.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-05-25T04:24:51.630 INFO:teuthology.orchestra.run.smithi112.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-05-25T04:24:51.631 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-05-25T04:24:51.631 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-05-25T04:24:51.631 INFO:teuthology.orchestra.run.smithi112.stdout: 2021-05-25T04:24:51.632 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2021-05-25T04:24:51.647 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: set os type and version on server 2021-05-25T04:24:51.647 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-25T04:24:51.859 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2021-05-25T04:24:51.914 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Linux" 2021-05-25T04:24:51.914 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="8 (Core)" 2021-05-25T04:24:51.915 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos" 2021-05-25T04:24:51.915 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora" 2021-05-25T04:24:51.915 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="8" 2021-05-25T04:24:51.915 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el8" 2021-05-25T04:24:51.916 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-05-25T04:24:51.916 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2021-05-25T04:24:51.916 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-05-25T04:24:51.916 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://www.centos.org/" 2021-05-25T04:24:51.916 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-05-25T04:24:51.917 INFO:teuthology.orchestra.run.smithi132.stdout: 2021-05-25T04:24:51.917 INFO:teuthology.orchestra.run.smithi132.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-05-25T04:24:51.917 INFO:teuthology.orchestra.run.smithi132.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-05-25T04:24:51.917 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-05-25T04:24:51.917 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-05-25T04:24:51.918 INFO:teuthology.orchestra.run.smithi132.stdout: 2021-05-25T04:24:51.918 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2021-05-25T04:24:51.992 INFO:teuthology.dispatcher.supervisor:Running job 6132876 2021-05-25T04:24:51.992 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-05-25_04:17:02-fs-pacific-distro-basic-smithi/6132876 --name teuthology-2021-05-25_04:17:02-fs-pacific-distro-basic-smithi --description fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/multimds_misc} -- /home/teuthworker/archive/teuthology-2021-05-25_04:17:02-fs-pacific-distro-basic-smithi/6132876/orig.config.yaml 2021-05-25T04:24:51.995 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-05-25_04:17:02-fs-pacific-distro-basic-smithi/6132876 2021-05-25T04:24:51.996 INFO:teuthology.dispatcher.supervisor:Job PID: 5930 2021-05-25T04:24:51.996 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-05-25T16:25:40.391 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-05-25T16:25:40.591 INFO:teuthology.kill:Killing Pids: {5930} 2021-05-25T16:25:41.620 INFO:teuthology.kill:Nuking machines: ['smithi112', 'smithi132'] 2021-05-25T16:25:42.526 INFO:teuthology.kill:2021-05-25 16:25:42,526.526 INFO:teuthology.nuke:targets: 2021-05-25T16:25:42.527 INFO:teuthology.kill: smithi112.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDks+wWs9yze2kG3hVxrAo7pP7qf6v4QchrwW0C8715DJpw2O8sYrDhGZBs84gHuAgFlyK0JS/Ciwnxna0ANwrhcVx31Pq+4GO4u4tA2fPl2Zt2Gr11vQKwYKf+wxbs55rKo9FM6D0yyIgA2vsdjdh7NwGvx2fWHY8BJBdSA4coCUZoM9c74GzMlr4krIs/XvkRgqT2M+jOQ7JIL74w2Jr8Gj3K6jzqOk6BqzJ1wuHlxSh3+hl5ePg+/aXP2sxTQr3gEkgz0222XipShSE7TzDJt0GAJkTi86LUcw5H8MnED+6Vf4jDsVRCQZ51AZpHJQ5aOJcML+wVUD5t/c3lLhYBtCAgnlf4XVgNCPx4Q3QxdFYC98RJGhobthkTw6wyzw+XCCiTunNvanm1wRajyUNJrk/4TqQSAJ5jm9g0ZPAA2ZLn+/uT/YWB2e55poIb/duP6Y34Z0F4wCOVkPhQORpUECVORpcH6UwySMt/GGZsAQmCJ2cx5JXTwQwPnzWM968= 2021-05-25T16:25:42.528 INFO:teuthology.kill: smithi132.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDngJkrKB/bPemtdyulZ5BPhnDuDR9xqJkbBl7f2GEihN2Cs5ULZF+vX5PED3CLUddNYHS0h9B1En+hrehUFWZeFEXvPv9hg1RS7aB97fBUkTRBX+y7MrkIFvElvGgkL6H91Xu4JZyN8r3A5jAf/QsXktgaO4U5GxTQsEEw9uO/xnJUQNb5MszkDS4LJ2Q68+B1CyGL82XFgwStu+yECTzBtKGK6L96x0+VrUHKtPAccciDgmbZgOpVzze3zr7vcDNjKJN5X4HjTvN2AJFm/CrGaxnv/QnFPAfu3o2pfcWZ5AVkL1LPpPvvv26S4ITSnn3guAC2dGxjkrdymH6HsPYxlHUPUJjjfwAJAoqdA3Py3Ok8w5x9UupgiqzDvnS6gDGDqdYIAiPqiV/c3d48n0B20da00qd6PXxRcisVXHK8EyoqhmvxqESFBiRj7HMeEfsc5b02J4+DZJBb7oFUlk3Fon6dCYJkHCMj+oxAld+YhdCuxO2qZAual0m3d/6TrWk= 2021-05-25T16:25:42.528 INFO:teuthology.kill:2021-05-25 16:25:42,527.527 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-25T16:25:42.529 INFO:teuthology.kill:2021-05-25 16:25:42,529.529 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-25T16:25:42.584 INFO:teuthology.kill:2021-05-25 16:25:42,584.584 INFO:teuthology.orchestra.console:Power off smithi112 2021-05-25T16:25:42.595 INFO:teuthology.kill:2021-05-25 16:25:42,595.595 INFO:teuthology.orchestra.console:Power off smithi132 2021-05-25T16:25:55.061 INFO:teuthology.kill:2021-05-25 16:25:55,060.060 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2021-05-25T16:25:55.066 INFO:teuthology.kill:2021-05-25 16:25:55,066.066 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2021-05-25T16:25:55.291 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2021-05-25T16:25:55.292 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2021-05-25T16:25:55.293 INFO:teuthology.misc:Compressing logs... 2021-05-25T16:25:55.293 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-25T16:26:29.963 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-05-25T16:26:30.023 INFO:teuthology.kill:No teuthology processes running 2021-05-25T16:26:31.059 INFO:teuthology.kill:Nuking machines: ['smithi112', 'smithi132'] 2021-05-25T16:26:31.948 INFO:teuthology.kill:2021-05-25 16:26:31,947.947 INFO:teuthology.nuke:targets: 2021-05-25T16:26:31.949 INFO:teuthology.kill: smithi112.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDks+wWs9yze2kG3hVxrAo7pP7qf6v4QchrwW0C8715DJpw2O8sYrDhGZBs84gHuAgFlyK0JS/Ciwnxna0ANwrhcVx31Pq+4GO4u4tA2fPl2Zt2Gr11vQKwYKf+wxbs55rKo9FM6D0yyIgA2vsdjdh7NwGvx2fWHY8BJBdSA4coCUZoM9c74GzMlr4krIs/XvkRgqT2M+jOQ7JIL74w2Jr8Gj3K6jzqOk6BqzJ1wuHlxSh3+hl5ePg+/aXP2sxTQr3gEkgz0222XipShSE7TzDJt0GAJkTi86LUcw5H8MnED+6Vf4jDsVRCQZ51AZpHJQ5aOJcML+wVUD5t/c3lLhYBtCAgnlf4XVgNCPx4Q3QxdFYC98RJGhobthkTw6wyzw+XCCiTunNvanm1wRajyUNJrk/4TqQSAJ5jm9g0ZPAA2ZLn+/uT/YWB2e55poIb/duP6Y34Z0F4wCOVkPhQORpUECVORpcH6UwySMt/GGZsAQmCJ2cx5JXTwQwPnzWM968= 2021-05-25T16:26:31.949 INFO:teuthology.kill: smithi132.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDngJkrKB/bPemtdyulZ5BPhnDuDR9xqJkbBl7f2GEihN2Cs5ULZF+vX5PED3CLUddNYHS0h9B1En+hrehUFWZeFEXvPv9hg1RS7aB97fBUkTRBX+y7MrkIFvElvGgkL6H91Xu4JZyN8r3A5jAf/QsXktgaO4U5GxTQsEEw9uO/xnJUQNb5MszkDS4LJ2Q68+B1CyGL82XFgwStu+yECTzBtKGK6L96x0+VrUHKtPAccciDgmbZgOpVzze3zr7vcDNjKJN5X4HjTvN2AJFm/CrGaxnv/QnFPAfu3o2pfcWZ5AVkL1LPpPvvv26S4ITSnn3guAC2dGxjkrdymH6HsPYxlHUPUJjjfwAJAoqdA3Py3Ok8w5x9UupgiqzDvnS6gDGDqdYIAiPqiV/c3d48n0B20da00qd6PXxRcisVXHK8EyoqhmvxqESFBiRj7HMeEfsc5b02J4+DZJBb7oFUlk3Fon6dCYJkHCMj+oxAld+YhdCuxO2qZAual0m3d/6TrWk= 2021-05-25T16:26:31.968 INFO:teuthology.kill:2021-05-25 16:26:31,968.968 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-25T16:26:31.971 INFO:teuthology.kill:2021-05-25 16:26:31,971.971 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-25T16:26:32.023 INFO:teuthology.kill:2021-05-25 16:26:32,022.022 INFO:teuthology.orchestra.console:Power off smithi112 2021-05-25T16:26:32.036 INFO:teuthology.kill:2021-05-25 16:26:32,036.036 INFO:teuthology.orchestra.console:Power off smithi132 2021-05-25T16:26:32.163 INFO:teuthology.kill:2021-05-25 16:26:32,163.163 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2021-05-25T16:26:32.176 INFO:teuthology.kill:2021-05-25 16:26:32,176.176 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2021-05-25T16:26:32.302 INFO:teuthology.kill:2021-05-25 16:26:32,301.301 INFO:teuthology.lock.ops:unlocked smithi112.front.sepia.ceph.com 2021-05-25T16:26:32.312 INFO:teuthology.kill:2021-05-25 16:26:32,312.312 INFO:teuthology.lock.ops:unlocked smithi132.front.sepia.ceph.com 2021-05-25T16:28:32.590 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-05-25T16:28:32.691 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-05-25T16:28:32.692 INFO:teuthology.nuke:Checking targets against current locks 2021-05-25T16:28:32.762 INFO:teuthology.nuke:Not nuking smithi112.front.sepia.ceph.com because description doesn't match 2021-05-25T16:28:32.763 INFO:teuthology.nuke:Not nuking smithi132.front.sepia.ceph.com because description doesn't match