2021-05-23T01:32:44.350 INFO:root:teuthology version: 1.1.0-d92ec140 2021-05-23T01:32:44.451 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi093.front.sepia.ceph.com' reimaging 2021-05-23T01:32:44.452 INFO:teuthology.lock.ops:Updating [ubuntu@smithi093.front.sepia.ceph.com]: reset os type and version on server 2021-05-23T01:32:44.452 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server 2021-05-23T01:32:44.467 INFO:teuthology.lock.ops:Node 'ubuntu@smithi093.front.sepia.ceph.com' reimaging is complete 2021-05-23T01:32:44.468 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging 2021-05-23T01:32:44.468 INFO:teuthology.lock.ops:Updating [ubuntu@smithi110.front.sepia.ceph.com]: reset os type and version on server 2021-05-23T01:32:44.468 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2021-05-23T01:32:44.484 INFO:teuthology.lock.ops:Node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging is complete 2021-05-23T01:32:44.661 INFO:teuthology.provision.fog.smithi093:Scheduling deploy of ubuntu 20.04 2021-05-23T01:32:44.666 INFO:teuthology.provision.fog.smithi110:Scheduling deploy of ubuntu 20.04 2021-05-23T01:32:45.042 INFO:teuthology.orchestra.console:Power off smithi110 2021-05-23T01:32:45.121 INFO:teuthology.orchestra.console:Power off smithi093 2021-05-23T01:32:45.182 INFO:teuthology.orchestra.console:Power off for smithi110 completed 2021-05-23T01:32:45.262 INFO:teuthology.orchestra.console:Power off for smithi093 completed 2021-05-23T01:32:45.283 INFO:teuthology.orchestra.console:Power on smithi110 2021-05-23T01:32:45.364 INFO:teuthology.orchestra.console:Power on smithi093 2021-05-23T01:32:49.537 INFO:teuthology.orchestra.console:Power on for smithi110 completed 2021-05-23T01:32:49.638 INFO:teuthology.provision.fog.smithi110:Waiting for deploy to finish 2021-05-23T01:32:53.733 INFO:teuthology.orchestra.console:Power on for smithi093 completed 2021-05-23T01:32:53.835 INFO:teuthology.provision.fog.smithi093:Waiting for deploy to finish 2021-05-23T01:37:47.297 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-05-23T01:37:47.791 INFO:teuthology.provision.fog.smithi110:Node is ready 2021-05-23T01:37:47.838 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110.front.sepia.ceph.com 2021-05-23T01:37:47.847 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110 smithi110.front.sepia.ceph.com smithi110 2021-05-23T01:37:47.991 INFO:teuthology.provision.fog.smithi110:Deploy complete! 2021-05-23T01:37:54.300 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-05-23T01:37:54.717 INFO:teuthology.provision.fog.smithi093:Node is ready 2021-05-23T01:37:54.724 INFO:teuthology.orchestra.run.smithi093.stdout:smithi093.front.sepia.ceph.com 2021-05-23T01:37:54.732 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93 smithi093.front.sepia.ceph.com smithi093 2021-05-23T01:37:54.946 INFO:teuthology.provision.fog.smithi093:Deploy complete! 2021-05-23T01:37:55.101 INFO:teuthology.lock.ops:Checking smithi093.front.sepia.ceph.com 2021-05-23T01:37:55.102 INFO:teuthology.lock.ops:New key found. Updating... 2021-05-23T01:37:55.122 INFO:teuthology.lock.ops:Checking smithi110.front.sepia.ceph.com 2021-05-23T01:37:55.122 INFO:teuthology.lock.ops:New key found. Updating... 2021-05-23T01:37:55.139 INFO:teuthology.lock.ops:Updating [smithi093.front.sepia.ceph.com]: set os type and version on server 2021-05-23T01:37:55.140 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-23T01:37:55.323 INFO:teuthology.orchestra.run.smithi093.stdout:x86_64 2021-05-23T01:37:55.330 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="Ubuntu" 2021-05-23T01:37:55.330 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-05-23T01:37:55.331 INFO:teuthology.orchestra.run.smithi093.stdout:ID=ubuntu 2021-05-23T01:37:55.331 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE=debian 2021-05-23T01:37:55.332 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-05-23T01:37:55.332 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="20.04" 2021-05-23T01:37:55.332 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://www.ubuntu.com/" 2021-05-23T01:37:55.333 INFO:teuthology.orchestra.run.smithi093.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-05-23T01:37:55.333 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-05-23T01:37:55.334 INFO:teuthology.orchestra.run.smithi093.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-05-23T01:37:55.334 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_CODENAME=focal 2021-05-23T01:37:55.335 INFO:teuthology.orchestra.run.smithi093.stdout:UBUNTU_CODENAME=focal 2021-05-23T01:37:55.336 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server 2021-05-23T01:37:55.356 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: set os type and version on server 2021-05-23T01:37:55.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-23T01:37:55.531 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2021-05-23T01:37:55.538 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="Ubuntu" 2021-05-23T01:37:55.538 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-05-23T01:37:55.538 INFO:teuthology.orchestra.run.smithi110.stdout:ID=ubuntu 2021-05-23T01:37:55.539 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE=debian 2021-05-23T01:37:55.539 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-05-23T01:37:55.539 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="20.04" 2021-05-23T01:37:55.540 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://www.ubuntu.com/" 2021-05-23T01:37:55.540 INFO:teuthology.orchestra.run.smithi110.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-05-23T01:37:55.540 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-05-23T01:37:55.541 INFO:teuthology.orchestra.run.smithi110.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-05-23T01:37:55.541 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_CODENAME=focal 2021-05-23T01:37:55.541 INFO:teuthology.orchestra.run.smithi110.stdout:UBUNTU_CODENAME=focal 2021-05-23T01:37:55.542 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2021-05-23T01:37:55.629 INFO:teuthology.dispatcher.supervisor:Running job 6129198 2021-05-23T01:37:55.630 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2021-05-23_00:57:16-fs-wip-kefu-testing-2021-05-22-2207-distro-basic-smithi/6129198 --name pdonnell-2021-05-23_00:57:16-fs-wip-kefu-testing-2021-05-22-2207-distro-basic-smithi --description fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} -- /home/teuthworker/archive/pdonnell-2021-05-23_00:57:16-fs-wip-kefu-testing-2021-05-22-2207-distro-basic-smithi/6129198/orig.config.yaml 2021-05-23T01:37:55.634 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2021-05-23_00:57:16-fs-wip-kefu-testing-2021-05-22-2207-distro-basic-smithi/6129198 2021-05-23T01:37:55.637 INFO:teuthology.dispatcher.supervisor:Job PID: 27417 2021-05-23T01:37:55.638 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-05-23T13:38:44.198 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-05-23T13:38:44.405 INFO:teuthology.kill:Killing Pids: {27417} 2021-05-23T13:38:45.373 INFO:teuthology.kill:Nuking machines: ['smithi093', 'smithi110'] 2021-05-23T13:38:46.189 INFO:teuthology.kill:2021-05-23 13:38:46,188.188 INFO:teuthology.nuke:targets: 2021-05-23T13:38:46.189 INFO:teuthology.kill: smithi093.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjr8NljxgoiEQfM0b1NobYYAJsGAlH788POUpSLQIzloo30yWsqwm/4CqoQv5CwWbZB1cMgOxwTHCrtSX3EJF8XqZxYb2RGnIOUfApRKPfhlqpuS9YVGcajNSZpCBrHK07JJDOXrGKZzORxGKJOT0IiepILUuBF5z/PhrhLZgdV2rdY+ahDTam3Iyhp5QRsJFl3yGRcrgp4Xt6Sfg9ZtFc131YvxotJoNXSZj41AIHSIaQ2L3MFzaFA/4ow8vf1laUamAXsxpbJsklQzTClJjh7OaI3bPCnnSahiqPJ5ZKiWN2TiqnTguwzobikGCCIu9goByh1mW2gFLFWyakx6iA0hWy0jia4m6mFu7AnUAw7iGYq0vzbMtGKLi9cl3OTaWoOX9N9mx+41aRaqEh9oWnNvIUdbUzcA1VZaBDhlRhu1rQ6m84094h5mcb/OV8KrfQHJKeET7+aOVJmcisTm0vKcYFssjAjPV/wUlnLdUheQ64axdmdflJiF0rtNqUNjM= 2021-05-23T13:38:46.190 INFO:teuthology.kill: smithi110.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp0Z8EKiCmLwhDwVeOEkVVN+u5AQPrnrdDbR4RQTAo4G6oNPlBjJrRyS7T7CSIJ1OrOhY/DFJ1D6cEQMkZVS9UFoNnZ01KvbKuIX2s8+aPlJWj6bZXjASI/sGljNV48Aeo04/3l7H6VoszFylWgRo2UMe0CTgQ1yMS7oRRWHeFsGNAbS602f8mWiJ+UfNrQDCWwVJ4dbsds0Tc04/rrkLPu8Fzk7pAlgt35zxcJ0CYc7/s3nmuK4gHaFGU66BNnydGlHIni+1eanDA+jYmmDHn/i+utjb2g0IuwOnpf5Rn3XV6nmdS+/QrIxebuurDq+gcaSWnorMkv3L+qr7G0O10pqg9bPHsvufgq/Gev1oi2XenXJ7k3AV/kTLxfmwBpBg00yLpQKoD25gq6HE5B/4uWruJ0ZjQTKHIXaFSyqZRycHX5vgdE9RhAfrZyq+LqXQTmM6v13rz6ZU57AHbaEsoXGs4uM4xsUnDXm54wNdU1EDo45TADS0bggzJCN5zupU= 2021-05-23T13:38:46.190 INFO:teuthology.kill:2021-05-23 13:38:46,188.188 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-23T13:38:46.191 INFO:teuthology.kill:2021-05-23 13:38:46,190.190 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-23T13:38:46.246 INFO:teuthology.kill:2021-05-23 13:38:46,246.246 INFO:teuthology.orchestra.console:Power off smithi093 2021-05-23T13:38:46.257 INFO:teuthology.kill:2021-05-23 13:38:46,257.257 INFO:teuthology.orchestra.console:Power off smithi110 2021-05-23T13:38:54.616 INFO:teuthology.kill:2021-05-23 13:38:54,616.616 INFO:teuthology.orchestra.console:Power off for smithi110 completed 2021-05-23T13:38:58.723 INFO:teuthology.kill:2021-05-23 13:38:58,723.723 INFO:teuthology.orchestra.console:Power off for smithi093 completed 2021-05-23T13:38:58.925 INFO:teuthology.task.internal:roles: ubuntu@smithi093.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2021-05-23T13:38:58.925 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2021-05-23T13:38:58.926 INFO:teuthology.misc:Compressing logs... 2021-05-23T13:38:58.926 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-23T13:39:58.981 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-05-23T13:39:59.039 INFO:teuthology.kill:No teuthology processes running 2021-05-23T13:39:59.987 INFO:teuthology.kill:Nuking machines: ['smithi093', 'smithi110'] 2021-05-23T13:40:00.835 INFO:teuthology.kill:2021-05-23 13:40:00,834.834 INFO:teuthology.nuke:targets: 2021-05-23T13:40:00.836 INFO:teuthology.kill: smithi093.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjr8NljxgoiEQfM0b1NobYYAJsGAlH788POUpSLQIzloo30yWsqwm/4CqoQv5CwWbZB1cMgOxwTHCrtSX3EJF8XqZxYb2RGnIOUfApRKPfhlqpuS9YVGcajNSZpCBrHK07JJDOXrGKZzORxGKJOT0IiepILUuBF5z/PhrhLZgdV2rdY+ahDTam3Iyhp5QRsJFl3yGRcrgp4Xt6Sfg9ZtFc131YvxotJoNXSZj41AIHSIaQ2L3MFzaFA/4ow8vf1laUamAXsxpbJsklQzTClJjh7OaI3bPCnnSahiqPJ5ZKiWN2TiqnTguwzobikGCCIu9goByh1mW2gFLFWyakx6iA0hWy0jia4m6mFu7AnUAw7iGYq0vzbMtGKLi9cl3OTaWoOX9N9mx+41aRaqEh9oWnNvIUdbUzcA1VZaBDhlRhu1rQ6m84094h5mcb/OV8KrfQHJKeET7+aOVJmcisTm0vKcYFssjAjPV/wUlnLdUheQ64axdmdflJiF0rtNqUNjM= 2021-05-23T13:40:00.836 INFO:teuthology.kill: smithi110.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCp0Z8EKiCmLwhDwVeOEkVVN+u5AQPrnrdDbR4RQTAo4G6oNPlBjJrRyS7T7CSIJ1OrOhY/DFJ1D6cEQMkZVS9UFoNnZ01KvbKuIX2s8+aPlJWj6bZXjASI/sGljNV48Aeo04/3l7H6VoszFylWgRo2UMe0CTgQ1yMS7oRRWHeFsGNAbS602f8mWiJ+UfNrQDCWwVJ4dbsds0Tc04/rrkLPu8Fzk7pAlgt35zxcJ0CYc7/s3nmuK4gHaFGU66BNnydGlHIni+1eanDA+jYmmDHn/i+utjb2g0IuwOnpf5Rn3XV6nmdS+/QrIxebuurDq+gcaSWnorMkv3L+qr7G0O10pqg9bPHsvufgq/Gev1oi2XenXJ7k3AV/kTLxfmwBpBg00yLpQKoD25gq6HE5B/4uWruJ0ZjQTKHIXaFSyqZRycHX5vgdE9RhAfrZyq+LqXQTmM6v13rz6ZU57AHbaEsoXGs4uM4xsUnDXm54wNdU1EDo45TADS0bggzJCN5zupU= 2021-05-23T13:40:00.855 INFO:teuthology.kill:2021-05-23 13:40:00,854.854 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-23T13:40:00.857 INFO:teuthology.kill:2021-05-23 13:40:00,857.857 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-23T13:40:00.911 INFO:teuthology.kill:2021-05-23 13:40:00,911.911 INFO:teuthology.orchestra.console:Power off smithi110 2021-05-23T13:40:00.924 INFO:teuthology.kill:2021-05-23 13:40:00,924.924 INFO:teuthology.orchestra.console:Power off smithi093 2021-05-23T13:40:01.052 INFO:teuthology.kill:2021-05-23 13:40:01,052.052 INFO:teuthology.orchestra.console:Power off for smithi110 completed 2021-05-23T13:40:01.063 INFO:teuthology.kill:2021-05-23 13:40:01,063.063 INFO:teuthology.orchestra.console:Power off for smithi093 completed 2021-05-23T13:40:01.188 INFO:teuthology.kill:2021-05-23 13:40:01,188.188 INFO:teuthology.lock.ops:unlocked smithi110.front.sepia.ceph.com 2021-05-23T13:40:01.199 INFO:teuthology.kill:2021-05-23 13:40:01,199.199 INFO:teuthology.lock.ops:unlocked smithi093.front.sepia.ceph.com 2021-05-23T13:42:01.446 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15