2021-06-01T05:52:12.150 INFO:root:teuthology version: 1.1.0-f359b10d 2021-06-01T05:52:12.258 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi046.front.sepia.ceph.com' reimaging 2021-06-01T05:52:12.259 INFO:teuthology.lock.ops:Updating [ubuntu@smithi046.front.sepia.ceph.com]: reset os type and version on server 2021-06-01T05:52:12.260 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2021-06-01T05:52:12.276 INFO:teuthology.lock.ops:Node 'ubuntu@smithi046.front.sepia.ceph.com' reimaging is complete 2021-06-01T05:52:12.277 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi061.front.sepia.ceph.com' reimaging 2021-06-01T05:52:12.277 INFO:teuthology.lock.ops:Updating [ubuntu@smithi061.front.sepia.ceph.com]: reset os type and version on server 2021-06-01T05:52:12.277 INFO:teuthology.lock.ops:Updating smithi061.front.sepia.ceph.com on lock server 2021-06-01T05:52:12.295 INFO:teuthology.lock.ops:Node 'ubuntu@smithi061.front.sepia.ceph.com' reimaging is complete 2021-06-01T05:52:12.514 INFO:teuthology.provision.fog.smithi061:Scheduling deploy of ubuntu 20.04 2021-06-01T05:52:12.531 INFO:teuthology.provision.fog.smithi046:Scheduling deploy of ubuntu 20.04 2021-06-01T05:52:12.934 INFO:teuthology.orchestra.console:Power off smithi061 2021-06-01T05:52:12.975 INFO:teuthology.orchestra.console:Power off smithi046 2021-06-01T05:52:13.084 INFO:teuthology.orchestra.console:Power off for smithi061 completed 2021-06-01T05:52:13.117 INFO:teuthology.orchestra.console:Power off for smithi046 completed 2021-06-01T05:52:13.186 INFO:teuthology.orchestra.console:Power on smithi061 2021-06-01T05:52:13.218 INFO:teuthology.orchestra.console:Power on smithi046 2021-06-01T05:52:17.442 INFO:teuthology.orchestra.console:Power on for smithi061 completed 2021-06-01T05:52:17.543 INFO:teuthology.provision.fog.smithi061:Waiting for deploy to finish 2021-06-01T05:52:21.583 INFO:teuthology.orchestra.console:Power on for smithi046 completed 2021-06-01T05:52:21.685 INFO:teuthology.provision.fog.smithi046:Waiting for deploy to finish 2021-06-01T05:57:17.458 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-01T05:57:17.910 INFO:teuthology.provision.fog.smithi046:Node is ready 2021-06-01T05:57:17.958 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046.front.sepia.ceph.com 2021-06-01T05:57:17.965 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2021-06-01T05:57:18.152 INFO:teuthology.provision.fog.smithi046:Deploy complete! 2021-06-01T05:57:31.164 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-01T05:57:31.584 INFO:teuthology.provision.fog.smithi061:Node is ready 2021-06-01T05:57:31.631 INFO:teuthology.orchestra.run.smithi061.stdout:smithi061.front.sepia.ceph.com 2021-06-01T05:57:31.638 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61 smithi061.front.sepia.ceph.com smithi061 2021-06-01T05:57:31.792 INFO:teuthology.provision.fog.smithi061:Deploy complete! 2021-06-01T05:57:31.952 INFO:teuthology.lock.ops:Checking smithi046.front.sepia.ceph.com 2021-06-01T05:57:31.953 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-01T05:57:31.973 INFO:teuthology.lock.ops:Checking smithi061.front.sepia.ceph.com 2021-06-01T05:57:31.973 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-01T05:57:31.988 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: set os type and version on server 2021-06-01T05:57:31.989 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-01T05:57:32.297 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2021-06-01T05:57:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="Ubuntu" 2021-06-01T05:57:32.304 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-06-01T05:57:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:ID=ubuntu 2021-06-01T05:57:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE=debian 2021-06-01T05:57:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-06-01T05:57:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="20.04" 2021-06-01T05:57:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://www.ubuntu.com/" 2021-06-01T05:57:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-06-01T05:57:32.306 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-06-01T05:57:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-06-01T05:57:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_CODENAME=focal 2021-06-01T05:57:32.307 INFO:teuthology.orchestra.run.smithi046.stdout:UBUNTU_CODENAME=focal 2021-06-01T05:57:32.308 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2021-06-01T05:57:32.324 INFO:teuthology.lock.ops:Updating [smithi061.front.sepia.ceph.com]: set os type and version on server 2021-06-01T05:57:32.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-01T05:57:32.502 INFO:teuthology.orchestra.run.smithi061.stdout:x86_64 2021-06-01T05:57:32.550 INFO:teuthology.orchestra.run.smithi061.stdout:NAME="Ubuntu" 2021-06-01T05:57:32.550 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-06-01T05:57:32.551 INFO:teuthology.orchestra.run.smithi061.stdout:ID=ubuntu 2021-06-01T05:57:32.551 INFO:teuthology.orchestra.run.smithi061.stdout:ID_LIKE=debian 2021-06-01T05:57:32.551 INFO:teuthology.orchestra.run.smithi061.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-06-01T05:57:32.551 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION_ID="20.04" 2021-06-01T05:57:32.551 INFO:teuthology.orchestra.run.smithi061.stdout:HOME_URL="https://www.ubuntu.com/" 2021-06-01T05:57:32.552 INFO:teuthology.orchestra.run.smithi061.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-06-01T05:57:32.552 INFO:teuthology.orchestra.run.smithi061.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-06-01T05:57:32.552 INFO:teuthology.orchestra.run.smithi061.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-06-01T05:57:32.552 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION_CODENAME=focal 2021-06-01T05:57:32.553 INFO:teuthology.orchestra.run.smithi061.stdout:UBUNTU_CODENAME=focal 2021-06-01T05:57:32.553 INFO:teuthology.lock.ops:Updating smithi061.front.sepia.ceph.com on lock server 2021-06-01T05:57:32.633 INFO:teuthology.dispatcher.supervisor:Running job 6144577 2021-06-01T05:57:32.634 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-06-01_04:17:03-fs-pacific-distro-basic-smithi/6144577 --name teuthology-2021-06-01_04:17:03-fs-pacific-distro-basic-smithi --description fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/metrics} -- /home/teuthworker/archive/teuthology-2021-06-01_04:17:03-fs-pacific-distro-basic-smithi/6144577/orig.config.yaml 2021-06-01T05:57:32.636 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-06-01_04:17:03-fs-pacific-distro-basic-smithi/6144577 2021-06-01T05:57:32.637 INFO:teuthology.dispatcher.supervisor:Job PID: 2184 2021-06-01T05:57:32.638 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-06-01T17:58:21.924 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-06-01T17:58:22.122 INFO:teuthology.kill:Killing Pids: {2184} 2021-06-01T17:58:23.189 INFO:teuthology.kill:Nuking machines: ['smithi046', 'smithi061'] 2021-06-01T17:58:24.080 INFO:teuthology.kill:2021-06-01 17:58:24,080.080 INFO:teuthology.nuke:targets: 2021-06-01T17:58:24.081 INFO:teuthology.kill: smithi046.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD7fF1Qf/bu6B7hR4WowcPByTMKUs8MuXGH5tfgChf3XHetmKFCluSQ0OSZeog+ORqF6NR0XMixiI+i4V41iv/xuEm2i1P0StRVC5pfd8U959dFXuZYmQX7jgzazi3o5PG0OTfkx1HNXrky2bN2xVvG3jB7gExJvzMPCejRz6tjKfK9T5OdU6emF+WHDfyLOowKuT8Agol4EtXnYIq88bGeaHFPSdXVKw42Ao4Ou+WZd5mzisSZu1CYjXE2Hfe+E8vLliZuTqukiI+1YRnFnkVoDwi8qNb46U14LvaAZD3koyAWVufrxN6psxehL28Xq++XyvwmI+9pSw1fhi06U7tVJDx6SoTdnESwpxPcAP8viK8Mk5xcaSEddRPVyotOaJmlwF2Nhnb+NaMeqhERv4UcDz5pwVb7+1QCFEoRjh+pToOlQbSBjV1eQu3FJCaMtMaoK/CYJIFxNq0XzxdblgXMW8U6988MKq+pWm8Oy/DVVLe4x5aFHm800gWIRdtaT5U= 2021-06-01T17:58:24.081 INFO:teuthology.kill: smithi061.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJm85TKfUMMouh5KIlY1XYr0E5Cram6QlDBH3WxiXBy/21sCAC4qNRuyKOPr5r8S4EQj6xYFguQdcj0u7iFORFY19gTmfOCOQrrKsGsBl3My4HnNv499uz3X1AU10TXqxBbIMGN2H89o+mFCDCI4KdmlJOEBqLrS8Ri/VUTn2077zpqKNm/nticaB6i2ii+y6c2zMvWbZF3osJCKBmGm00MRZji+7pQcUJvVMtv5GTJQHkXqhnr5FDdy6CuFZhwsyDBaHY8p7BSwzg8pjj0nfM2TZaPs/i5x2ovRTRv26tqPw2i4MrQsE2dPn0jbyP4M+cqr4DY9paaWffkBgEt4Ifw7DZUSnFC33I7QflELXi1HUwSoWtCudHaawEUazDy8CULsIKgsBrDFmMkiJZt9qqFrrEBZykoWBXKLT3fTWOy0MLtJg3aL1AAa9aSBDP94xhIkQLTku0uSb6s5nifoIxdHgDKv65tcrTEgK7BxLGr/Zpr9MvjkIUejh9ba1N65k= 2021-06-01T17:58:24.082 INFO:teuthology.kill:2021-06-01 17:58:24,080.080 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-01T17:58:24.083 INFO:teuthology.kill:2021-06-01 17:58:24,083.083 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-01T17:58:24.146 INFO:teuthology.kill:2021-06-01 17:58:24,145.145 INFO:teuthology.orchestra.console:Power off smithi046 2021-06-01T17:58:24.159 INFO:teuthology.kill:2021-06-01 17:58:24,158.158 INFO:teuthology.orchestra.console:Power off smithi061 2021-06-01T17:58:32.512 INFO:teuthology.kill:2021-06-01 17:58:32,512.512 INFO:teuthology.orchestra.console:Power off for smithi046 completed 2021-06-01T17:58:36.627 INFO:teuthology.kill:2021-06-01 17:58:36,627.627 INFO:teuthology.orchestra.console:Power off for smithi061 completed 2021-06-01T17:58:36.844 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2021-06-01T17:58:36.844 INFO:teuthology.task.internal:roles: ubuntu@smithi061.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2021-06-01T17:58:36.845 INFO:teuthology.misc:Compressing logs... 2021-06-01T17:58:36.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-01T17:59:11.311 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-06-01T17:59:11.376 INFO:teuthology.kill:No teuthology processes running 2021-06-01T17:59:12.436 INFO:teuthology.kill:Nuking machines: ['smithi046', 'smithi061'] 2021-06-01T17:59:13.377 INFO:teuthology.kill:2021-06-01 17:59:13,377.377 INFO:teuthology.nuke:targets: 2021-06-01T17:59:13.378 INFO:teuthology.kill: smithi046.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD7fF1Qf/bu6B7hR4WowcPByTMKUs8MuXGH5tfgChf3XHetmKFCluSQ0OSZeog+ORqF6NR0XMixiI+i4V41iv/xuEm2i1P0StRVC5pfd8U959dFXuZYmQX7jgzazi3o5PG0OTfkx1HNXrky2bN2xVvG3jB7gExJvzMPCejRz6tjKfK9T5OdU6emF+WHDfyLOowKuT8Agol4EtXnYIq88bGeaHFPSdXVKw42Ao4Ou+WZd5mzisSZu1CYjXE2Hfe+E8vLliZuTqukiI+1YRnFnkVoDwi8qNb46U14LvaAZD3koyAWVufrxN6psxehL28Xq++XyvwmI+9pSw1fhi06U7tVJDx6SoTdnESwpxPcAP8viK8Mk5xcaSEddRPVyotOaJmlwF2Nhnb+NaMeqhERv4UcDz5pwVb7+1QCFEoRjh+pToOlQbSBjV1eQu3FJCaMtMaoK/CYJIFxNq0XzxdblgXMW8U6988MKq+pWm8Oy/DVVLe4x5aFHm800gWIRdtaT5U= 2021-06-01T17:59:13.378 INFO:teuthology.kill: smithi061.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJm85TKfUMMouh5KIlY1XYr0E5Cram6QlDBH3WxiXBy/21sCAC4qNRuyKOPr5r8S4EQj6xYFguQdcj0u7iFORFY19gTmfOCOQrrKsGsBl3My4HnNv499uz3X1AU10TXqxBbIMGN2H89o+mFCDCI4KdmlJOEBqLrS8Ri/VUTn2077zpqKNm/nticaB6i2ii+y6c2zMvWbZF3osJCKBmGm00MRZji+7pQcUJvVMtv5GTJQHkXqhnr5FDdy6CuFZhwsyDBaHY8p7BSwzg8pjj0nfM2TZaPs/i5x2ovRTRv26tqPw2i4MrQsE2dPn0jbyP4M+cqr4DY9paaWffkBgEt4Ifw7DZUSnFC33I7QflELXi1HUwSoWtCudHaawEUazDy8CULsIKgsBrDFmMkiJZt9qqFrrEBZykoWBXKLT3fTWOy0MLtJg3aL1AAa9aSBDP94xhIkQLTku0uSb6s5nifoIxdHgDKv65tcrTEgK7BxLGr/Zpr9MvjkIUejh9ba1N65k= 2021-06-01T17:59:13.397 INFO:teuthology.kill:2021-06-01 17:59:13,397.397 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-01T17:59:13.400 INFO:teuthology.kill:2021-06-01 17:59:13,400.400 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-01T17:59:13.458 INFO:teuthology.kill:2021-06-01 17:59:13,458.458 INFO:teuthology.orchestra.console:Power off smithi061 2021-06-01T17:59:13.472 INFO:teuthology.kill:2021-06-01 17:59:13,472.472 INFO:teuthology.orchestra.console:Power off smithi046 2021-06-01T17:59:13.605 INFO:teuthology.kill:2021-06-01 17:59:13,604.604 INFO:teuthology.orchestra.console:Power off for smithi061 completed 2021-06-01T17:59:13.618 INFO:teuthology.kill:2021-06-01 17:59:13,618.618 INFO:teuthology.orchestra.console:Power off for smithi046 completed 2021-06-01T17:59:13.749 INFO:teuthology.kill:2021-06-01 17:59:13,748.748 INFO:teuthology.lock.ops:unlocked smithi061.front.sepia.ceph.com 2021-06-01T17:59:13.754 INFO:teuthology.kill:2021-06-01 17:59:13,754.754 INFO:teuthology.lock.ops:unlocked smithi046.front.sepia.ceph.com 2021-06-01T18:01:14.046 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-06-01T18:01:14.135 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-06-01T18:01:14.136 INFO:teuthology.nuke:Checking targets against current locks 2021-06-01T18:01:14.206 INFO:teuthology.nuke:Not nuking smithi061.front.sepia.ceph.com because description doesn't match 2021-06-01T18:01:14.207 INFO:teuthology.nuke:Not nuking smithi046.front.sepia.ceph.com because description doesn't match