2021-05-23T02:11:52.616 INFO:root:teuthology version: 1.1.0-d92ec140 2021-05-23T02:11:52.706 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi045.front.sepia.ceph.com' reimaging 2021-05-23T02:11:52.707 INFO:teuthology.lock.ops:Updating [ubuntu@smithi045.front.sepia.ceph.com]: reset os type and version on server 2021-05-23T02:11:52.707 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2021-05-23T02:11:52.721 INFO:teuthology.lock.ops:Node 'ubuntu@smithi045.front.sepia.ceph.com' reimaging is complete 2021-05-23T02:11:52.721 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi191.front.sepia.ceph.com' reimaging 2021-05-23T02:11:52.721 INFO:teuthology.lock.ops:Updating [ubuntu@smithi191.front.sepia.ceph.com]: reset os type and version on server 2021-05-23T02:11:52.722 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2021-05-23T02:11:52.736 INFO:teuthology.lock.ops:Node 'ubuntu@smithi191.front.sepia.ceph.com' reimaging is complete 2021-05-23T02:11:52.946 INFO:teuthology.provision.fog.smithi045:Scheduling deploy of ubuntu 20.04 2021-05-23T02:11:52.958 INFO:teuthology.provision.fog.smithi191:Scheduling deploy of ubuntu 20.04 2021-05-23T02:11:53.328 INFO:teuthology.orchestra.console:Power off smithi045 2021-05-23T02:11:53.423 INFO:teuthology.orchestra.console:Power off smithi191 2021-05-23T02:12:01.693 INFO:teuthology.orchestra.console:Power off for smithi045 completed 2021-05-23T02:12:01.794 INFO:teuthology.orchestra.console:Power on smithi045 2021-05-23T02:12:05.901 INFO:teuthology.orchestra.console:Power off for smithi191 completed 2021-05-23T02:12:06.003 INFO:teuthology.orchestra.console:Power on smithi191 2021-05-23T02:12:10.157 INFO:teuthology.orchestra.console:Power on for smithi045 completed 2021-05-23T02:12:10.253 INFO:teuthology.orchestra.console:Power on for smithi191 completed 2021-05-23T02:12:10.259 INFO:teuthology.provision.fog.smithi045:Waiting for deploy to finish 2021-05-23T02:12:10.354 INFO:teuthology.provision.fog.smithi191:Waiting for deploy to finish 2021-05-23T02:16:39.130 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-05-23T02:16:39.612 INFO:teuthology.provision.fog.smithi045:Node is ready 2021-05-23T02:16:39.620 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045.front.sepia.ceph.com 2021-05-23T02:16:39.627 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45 smithi045.front.sepia.ceph.com smithi045 2021-05-23T02:16:39.802 INFO:teuthology.provision.fog.smithi045:Deploy complete! 2021-05-23T02:17:17.953 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-05-23T02:17:18.429 INFO:teuthology.provision.fog.smithi191:Node is ready 2021-05-23T02:17:18.436 INFO:teuthology.orchestra.run.smithi191.stdout:smithi191.front.sepia.ceph.com 2021-05-23T02:17:18.443 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.15.191 smithi191.front.sepia.ceph.com smithi191 2021-05-23T02:17:18.677 INFO:teuthology.provision.fog.smithi191:Deploy complete! 2021-05-23T02:17:18.841 INFO:teuthology.lock.ops:Checking smithi045.front.sepia.ceph.com 2021-05-23T02:17:18.842 INFO:teuthology.lock.ops:New key found. Updating... 2021-05-23T02:17:18.860 INFO:teuthology.lock.ops:Checking smithi191.front.sepia.ceph.com 2021-05-23T02:17:18.860 INFO:teuthology.lock.ops:New key found. Updating... 2021-05-23T02:17:18.877 INFO:teuthology.lock.ops:Updating [smithi045.front.sepia.ceph.com]: set os type and version on server 2021-05-23T02:17:18.878 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-23T02:17:19.135 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2021-05-23T02:17:19.141 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="Ubuntu" 2021-05-23T02:17:19.142 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-05-23T02:17:19.142 INFO:teuthology.orchestra.run.smithi045.stdout:ID=ubuntu 2021-05-23T02:17:19.142 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE=debian 2021-05-23T02:17:19.142 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-05-23T02:17:19.143 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="20.04" 2021-05-23T02:17:19.143 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://www.ubuntu.com/" 2021-05-23T02:17:19.143 INFO:teuthology.orchestra.run.smithi045.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-05-23T02:17:19.143 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-05-23T02:17:19.144 INFO:teuthology.orchestra.run.smithi045.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-05-23T02:17:19.144 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_CODENAME=focal 2021-05-23T02:17:19.144 INFO:teuthology.orchestra.run.smithi045.stdout:UBUNTU_CODENAME=focal 2021-05-23T02:17:19.145 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2021-05-23T02:17:19.160 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: set os type and version on server 2021-05-23T02:17:19.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-23T02:17:19.329 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2021-05-23T02:17:19.336 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="Ubuntu" 2021-05-23T02:17:19.336 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-05-23T02:17:19.336 INFO:teuthology.orchestra.run.smithi191.stdout:ID=ubuntu 2021-05-23T02:17:19.337 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE=debian 2021-05-23T02:17:19.337 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-05-23T02:17:19.337 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="20.04" 2021-05-23T02:17:19.337 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://www.ubuntu.com/" 2021-05-23T02:17:19.337 INFO:teuthology.orchestra.run.smithi191.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-05-23T02:17:19.338 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-05-23T02:17:19.338 INFO:teuthology.orchestra.run.smithi191.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-05-23T02:17:19.338 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_CODENAME=focal 2021-05-23T02:17:19.339 INFO:teuthology.orchestra.run.smithi191.stdout:UBUNTU_CODENAME=focal 2021-05-23T02:17:19.339 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2021-05-23T02:17:19.421 INFO:teuthology.dispatcher.supervisor:Running job 6129245 2021-05-23T02:17:19.421 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2021-05-23_00:57:16-fs-wip-kefu-testing-2021-05-22-2207-distro-basic-smithi/6129245 --name pdonnell-2021-05-23_00:57:16-fs-wip-kefu-testing-2021-05-22-2207-distro-basic-smithi --description fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm} -- /home/teuthworker/archive/pdonnell-2021-05-23_00:57:16-fs-wip-kefu-testing-2021-05-22-2207-distro-basic-smithi/6129245/orig.config.yaml 2021-05-23T02:17:19.425 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2021-05-23_00:57:16-fs-wip-kefu-testing-2021-05-22-2207-distro-basic-smithi/6129245 2021-05-23T02:17:19.427 INFO:teuthology.dispatcher.supervisor:Job PID: 20900 2021-05-23T02:17:19.427 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-05-23T14:18:08.022 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-05-23T14:18:08.227 INFO:teuthology.kill:Killing Pids: {20900} 2021-05-23T14:18:09.253 INFO:teuthology.kill:Nuking machines: ['smithi045', 'smithi191'] 2021-05-23T14:18:10.153 INFO:teuthology.kill:2021-05-23 14:18:10,153.153 INFO:teuthology.nuke:targets: 2021-05-23T14:18:10.154 INFO:teuthology.kill: smithi045.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmXTAmuMVK93c0/WDx4LmkIFEDpkORP/8UYmRzNG9kJl1vs0Ppa40wawC274pRweulP4XbJ5l8cJ9o7cnbdnMlw5u9HPitm7BDhVnHBgLWPmytDuCloAekXMSC85vnGZCT69iEI1Gekyn57dkm1r7c//rwNZsLhbTa7iBHhOmqJxgtREXQ9R2/1JjPMMFa+pLUZVytcFLDxRVlQXjCFgjK3OVMhwgT8la9Z5iFni46yfV9puYqrppr/T4t1VCgo57IOpBIDak/kROFEE77Ijo/ro52dIw/Y/rJKdtD25x3mmAMhFAbGmNr5qRy10ZGtJGjrJcgvjT3CHtm95CJqYWYpRAvdRNYuiOOZtw8Ugt49ds5/GGDEqr85iGpEWTz+TUJVFE3Bshi83/mC9PcgnlAUbI8MOOmMLYplIevc6NmEkFou+vmaXPmNh32b7RgzAxlmsftp6Ic6X5iIRAH/0Mc2zZVxl+QgaQLctsm5kM6lkGUoK0CnvH3LddJNfIUARU= 2021-05-23T14:18:10.155 INFO:teuthology.kill: smithi191.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAJSv0V6Skm9TYm4rCUf3EVKIBTJxqQZEnxABcsBiKC4QLwAWxPaQLvyyMnNJPSCiFoSDaXhjtJKyN+31TznJKM1XhyC2dNQK72C0JVD10jEL5o2qsJnQIMd+UDPWBIhwprEjdEdZ/0IBzhvumBfl8uRaUXnEryiQiT6gsAzq8VxNqC3Qmeud0SohCF/+pIqzV1/+p8leFS5mCmx/dftVeYD+S92tO25j6s05hbUGg2IoCqx70Vr45H9qec2zqq382+rVu9aMLz3znbi/gvARipkDVULnOKIhaUUghSG8aOIEEJ+RHAOBcULz7lJBY2CYJDLw2GncF4x57lbHhTQWHvnRKrJxWSKSfHB8Xa3rk4Z1qKLiEo5jxy3wr/ge7eKsaKuMZqtH5VrJ7UDkodO3Ve/FgpwE0S6pXXr99pVcUXoGJP9nmZuzK6eEyEZ+z+sR/52B773mwkD/WQhG0xVdc5EmGujand0M66tO9S5ZsAUwqdHQ7QMuV5gormI4qIh8= 2021-05-23T14:18:10.155 INFO:teuthology.kill:2021-05-23 14:18:10,154.154 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-23T14:18:10.156 INFO:teuthology.kill:2021-05-23 14:18:10,156.156 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-23T14:18:10.210 INFO:teuthology.kill:2021-05-23 14:18:10,210.210 INFO:teuthology.orchestra.console:Power off smithi045 2021-05-23T14:18:10.222 INFO:teuthology.kill:2021-05-23 14:18:10,222.222 INFO:teuthology.orchestra.console:Power off smithi191 2021-05-23T14:18:18.578 INFO:teuthology.kill:2021-05-23 14:18:18,577.577 INFO:teuthology.orchestra.console:Power off for smithi045 completed 2021-05-23T14:18:18.583 INFO:teuthology.kill:2021-05-23 14:18:18,583.583 INFO:teuthology.orchestra.console:Power off for smithi191 completed 2021-05-23T14:18:18.832 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2021-05-23T14:18:18.833 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2021-05-23T14:18:18.833 INFO:teuthology.misc:Compressing logs... 2021-05-23T14:18:18.834 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-23T14:19:18.898 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-05-23T14:19:18.969 INFO:teuthology.kill:No teuthology processes running 2021-05-23T14:19:19.998 INFO:teuthology.kill:Nuking machines: ['smithi045', 'smithi191'] 2021-05-23T14:19:20.875 INFO:teuthology.kill:2021-05-23 14:19:20,875.875 INFO:teuthology.nuke:targets: 2021-05-23T14:19:20.876 INFO:teuthology.kill: smithi045.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmXTAmuMVK93c0/WDx4LmkIFEDpkORP/8UYmRzNG9kJl1vs0Ppa40wawC274pRweulP4XbJ5l8cJ9o7cnbdnMlw5u9HPitm7BDhVnHBgLWPmytDuCloAekXMSC85vnGZCT69iEI1Gekyn57dkm1r7c//rwNZsLhbTa7iBHhOmqJxgtREXQ9R2/1JjPMMFa+pLUZVytcFLDxRVlQXjCFgjK3OVMhwgT8la9Z5iFni46yfV9puYqrppr/T4t1VCgo57IOpBIDak/kROFEE77Ijo/ro52dIw/Y/rJKdtD25x3mmAMhFAbGmNr5qRy10ZGtJGjrJcgvjT3CHtm95CJqYWYpRAvdRNYuiOOZtw8Ugt49ds5/GGDEqr85iGpEWTz+TUJVFE3Bshi83/mC9PcgnlAUbI8MOOmMLYplIevc6NmEkFou+vmaXPmNh32b7RgzAxlmsftp6Ic6X5iIRAH/0Mc2zZVxl+QgaQLctsm5kM6lkGUoK0CnvH3LddJNfIUARU= 2021-05-23T14:19:20.877 INFO:teuthology.kill: smithi191.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAJSv0V6Skm9TYm4rCUf3EVKIBTJxqQZEnxABcsBiKC4QLwAWxPaQLvyyMnNJPSCiFoSDaXhjtJKyN+31TznJKM1XhyC2dNQK72C0JVD10jEL5o2qsJnQIMd+UDPWBIhwprEjdEdZ/0IBzhvumBfl8uRaUXnEryiQiT6gsAzq8VxNqC3Qmeud0SohCF/+pIqzV1/+p8leFS5mCmx/dftVeYD+S92tO25j6s05hbUGg2IoCqx70Vr45H9qec2zqq382+rVu9aMLz3znbi/gvARipkDVULnOKIhaUUghSG8aOIEEJ+RHAOBcULz7lJBY2CYJDLw2GncF4x57lbHhTQWHvnRKrJxWSKSfHB8Xa3rk4Z1qKLiEo5jxy3wr/ge7eKsaKuMZqtH5VrJ7UDkodO3Ve/FgpwE0S6pXXr99pVcUXoGJP9nmZuzK6eEyEZ+z+sR/52B773mwkD/WQhG0xVdc5EmGujand0M66tO9S5ZsAUwqdHQ7QMuV5gormI4qIh8= 2021-05-23T14:19:20.895 INFO:teuthology.kill:2021-05-23 14:19:20,895.895 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-23T14:19:20.898 INFO:teuthology.kill:2021-05-23 14:19:20,898.898 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-23T14:19:20.954 INFO:teuthology.kill:2021-05-23 14:19:20,953.953 INFO:teuthology.orchestra.console:Power off smithi045 2021-05-23T14:19:20.965 INFO:teuthology.kill:2021-05-23 14:19:20,965.965 INFO:teuthology.orchestra.console:Power off smithi191 2021-05-23T14:19:21.093 INFO:teuthology.kill:2021-05-23 14:19:21,093.093 INFO:teuthology.orchestra.console:Power off for smithi045 completed 2021-05-23T14:19:21.104 INFO:teuthology.kill:2021-05-23 14:19:21,104.104 INFO:teuthology.orchestra.console:Power off for smithi191 completed 2021-05-23T14:19:21.232 INFO:teuthology.kill:2021-05-23 14:19:21,231.231 INFO:teuthology.lock.ops:unlocked smithi045.front.sepia.ceph.com 2021-05-23T14:19:21.241 INFO:teuthology.kill:2021-05-23 14:19:21,241.241 INFO:teuthology.lock.ops:unlocked smithi191.front.sepia.ceph.com 2021-05-23T14:21:21.491 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15