2021-06-03T14:33:00.351 INFO:root:teuthology version: 1.1.0-f359b10d 2021-06-03T14:33:00.450 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi164.front.sepia.ceph.com' reimaging 2021-06-03T14:33:00.451 INFO:teuthology.lock.ops:Updating [ubuntu@smithi164.front.sepia.ceph.com]: reset os type and version on server 2021-06-03T14:33:00.452 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2021-06-03T14:33:00.468 INFO:teuthology.lock.ops:Node 'ubuntu@smithi164.front.sepia.ceph.com' reimaging is complete 2021-06-03T14:33:00.468 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging 2021-06-03T14:33:00.468 INFO:teuthology.lock.ops:Updating [ubuntu@smithi170.front.sepia.ceph.com]: reset os type and version on server 2021-06-03T14:33:00.469 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2021-06-03T14:33:00.487 INFO:teuthology.lock.ops:Node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging is complete 2021-06-03T14:33:00.669 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of ubuntu 20.04 2021-06-03T14:33:00.689 INFO:teuthology.provision.fog.smithi170:Scheduling deploy of ubuntu 20.04 2021-06-03T14:33:01.011 INFO:teuthology.orchestra.console:Power off smithi164 2021-06-03T14:33:01.046 INFO:teuthology.orchestra.console:Power off smithi170 2021-06-03T14:33:13.492 INFO:teuthology.orchestra.console:Power off for smithi164 completed 2021-06-03T14:33:13.520 INFO:teuthology.orchestra.console:Power off for smithi170 completed 2021-06-03T14:33:13.593 INFO:teuthology.orchestra.console:Power on smithi164 2021-06-03T14:33:13.621 INFO:teuthology.orchestra.console:Power on smithi170 2021-06-03T14:33:17.844 INFO:teuthology.orchestra.console:Power on for smithi164 completed 2021-06-03T14:33:17.882 INFO:teuthology.orchestra.console:Power on for smithi170 completed 2021-06-03T14:33:17.945 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish 2021-06-03T14:33:17.984 INFO:teuthology.provision.fog.smithi170:Waiting for deploy to finish 2021-06-03T14:38:21.465 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-03T14:38:21.924 INFO:teuthology.provision.fog.smithi170:Node is ready 2021-06-03T14:38:21.932 INFO:teuthology.orchestra.run.smithi170.stdout:smithi170.front.sepia.ceph.com 2021-06-03T14:38:21.979 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170 smithi170.front.sepia.ceph.com smithi170 2021-06-03T14:38:22.171 INFO:teuthology.provision.fog.smithi170:Deploy complete! 2021-06-03T14:38:30.461 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-03T14:38:30.893 INFO:teuthology.provision.fog.smithi164:Node is ready 2021-06-03T14:38:30.900 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164.front.sepia.ceph.com 2021-06-03T14:38:30.950 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164 2021-06-03T14:38:31.147 INFO:teuthology.provision.fog.smithi164:Deploy complete! 2021-06-03T14:38:31.308 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com 2021-06-03T14:38:31.309 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-03T14:38:31.326 INFO:teuthology.lock.ops:Checking smithi170.front.sepia.ceph.com 2021-06-03T14:38:31.326 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-03T14:38:31.338 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server 2021-06-03T14:38:31.339 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-03T14:38:31.515 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2021-06-03T14:38:31.519 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="Ubuntu" 2021-06-03T14:38:31.520 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-06-03T14:38:31.520 INFO:teuthology.orchestra.run.smithi164.stdout:ID=ubuntu 2021-06-03T14:38:31.520 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE=debian 2021-06-03T14:38:31.520 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-06-03T14:38:31.521 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="20.04" 2021-06-03T14:38:31.521 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.ubuntu.com/" 2021-06-03T14:38:31.521 INFO:teuthology.orchestra.run.smithi164.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-06-03T14:38:31.521 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-06-03T14:38:31.521 INFO:teuthology.orchestra.run.smithi164.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-06-03T14:38:31.522 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_CODENAME=focal 2021-06-03T14:38:31.522 INFO:teuthology.orchestra.run.smithi164.stdout:UBUNTU_CODENAME=focal 2021-06-03T14:38:31.523 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2021-06-03T14:38:31.538 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: set os type and version on server 2021-06-03T14:38:31.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-03T14:38:31.714 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2021-06-03T14:38:31.762 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Ubuntu" 2021-06-03T14:38:31.763 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-06-03T14:38:31.763 INFO:teuthology.orchestra.run.smithi170.stdout:ID=ubuntu 2021-06-03T14:38:31.763 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE=debian 2021-06-03T14:38:31.764 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-06-03T14:38:31.764 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="20.04" 2021-06-03T14:38:31.764 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.ubuntu.com/" 2021-06-03T14:38:31.764 INFO:teuthology.orchestra.run.smithi170.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-06-03T14:38:31.764 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-06-03T14:38:31.765 INFO:teuthology.orchestra.run.smithi170.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-06-03T14:38:31.765 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_CODENAME=focal 2021-06-03T14:38:31.765 INFO:teuthology.orchestra.run.smithi170.stdout:UBUNTU_CODENAME=focal 2021-06-03T14:38:31.766 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2021-06-03T14:38:31.850 INFO:teuthology.dispatcher.supervisor:Running job 6147193 2021-06-03T14:38:31.851 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-06-02_18:29:36-fs-wip-yuri3-testing-2021-06-02-0826-pacific-distro-basic-smithi/6147193 --name yuriw-2021-06-02_18:29:36-fs-wip-yuri3-testing-2021-06-02-0826-pacific-distro-basic-smithi --description fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cap-flush} -- /home/teuthworker/archive/yuriw-2021-06-02_18:29:36-fs-wip-yuri3-testing-2021-06-02-0826-pacific-distro-basic-smithi/6147193/orig.config.yaml 2021-06-03T14:38:31.854 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-06-02_18:29:36-fs-wip-yuri3-testing-2021-06-02-0826-pacific-distro-basic-smithi/6147193 2021-06-03T14:38:31.854 INFO:teuthology.dispatcher.supervisor:Job PID: 8599 2021-06-03T14:38:31.855 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-06-04T02:39:21.227 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-06-04T02:39:21.446 INFO:teuthology.kill:Killing Pids: {8599} 2021-06-04T02:39:22.734 INFO:teuthology.kill:Nuking machines: ['smithi164', 'smithi170'] 2021-06-04T02:39:23.744 INFO:teuthology.kill:2021-06-04 02:39:23,744.744 INFO:teuthology.nuke:targets: 2021-06-04T02:39:23.746 INFO:teuthology.kill: smithi164.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5B1ejR+SlKbKtHezwL7BrSBbjnLj/W+yfHGzqhD/aEVgjduL1PBIwcsV9tCAlKFyG98m3rvFqOIheG86gsZQIUtH6/bd3UhFTbxzTl4EMW/I8lsEglmtqs35+g11pamnLXCOlW+iYnGWJdk/PQ3zRyjMNIDxBII216rH99VhGOLM5ObJIkw1XXxXuGx9s7d5NEFcNpuF6VBgobcjQCVTKH/9qzGUfwxbyMfmHR4LfE9eu9Qccd4G7fCEMryLagHebtChI4CWTUXUs2GqybsKiKSkkelavw3WN7E2oqaVlzHDcV5bVH67VSpuEBKWL3pV2TKCV4d4bv5tHeCL0V9z1ttJn3BQMSdP6v9RDx5aoH+UOJTBPrNA+N7OEKG91avxxH5GYzhHG354idPrSwZKdplYg1UI9T2+NJMgdpvL/IrTOOEFmyVaRvMLe4PMt9gx1qbPo0fan1ayxeDn659RB5EouhNVs4EIX4m9x2MB3z6SsAxIanDpq/sSnYwNmDZE= 2021-06-04T02:39:23.746 INFO:teuthology.kill: smithi170.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1e+HSlbTazgxj8wVSjYsu31FjNaEyTEZ2514aiOTBUEcW2wbtBOh/yc6h9dbQbme4IAQ5yru9mNSk0Cr0kWo/L+UspvDjwTZM7zkmp2eTDqydYFLYsz8kggcAr9BUsE7tVZiQ00Zlz0Yzp6sXdQKC1yJZKU1tU4UccdBfOjrl0tRcwsXYoShNcN3eL2VXauy6M5Cwfg+5BGC6PBH49U9wwiahLWvCdcGwxLNZCWiQRQIL9iAFZUF0dHCuL9x98BaS/kryQMechDnGTgSMvQGbc8cS4zZ7Hr++bs37AxmI7K1rJZRo2tDd1X3b5ep1Pak8QiaHlSy398djZh5eOqIFHb0MhVXOMwvv91gZk0dQu0kfsQXSOB43tOyc8CshFxqBNUJlsY7VLAq8rrTgmBkBj5dMUW0LJgfs2LH82gaEFNvw+z3f0Oh7TEszRxa1jIOKYIkxvte5qenNqjgqR7xeS1Nv06tXtwAEAiIB9vsVwnC7mIeezxJg5gxDuBSoHq8= 2021-06-04T02:39:23.746 INFO:teuthology.kill:2021-06-04 02:39:23,744.744 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-04T02:39:23.748 INFO:teuthology.kill:2021-06-04 02:39:23,747.747 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-04T02:39:23.809 INFO:teuthology.kill:2021-06-04 02:39:23,809.809 INFO:teuthology.orchestra.console:Power off smithi164 2021-06-04T02:39:23.823 INFO:teuthology.kill:2021-06-04 02:39:23,823.823 INFO:teuthology.orchestra.console:Power off smithi170 2021-06-04T02:39:32.182 INFO:teuthology.kill:2021-06-04 02:39:32,182.182 INFO:teuthology.orchestra.console:Power off for smithi164 completed 2021-06-04T02:39:32.189 INFO:teuthology.kill:2021-06-04 02:39:32,189.189 INFO:teuthology.orchestra.console:Power off for smithi170 completed 2021-06-04T02:39:32.447 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2021-06-04T02:39:32.448 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2021-06-04T02:39:32.448 INFO:teuthology.misc:Compressing logs... 2021-06-04T02:39:32.449 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-04T02:40:07.115 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-06-04T02:40:07.185 INFO:teuthology.kill:No teuthology processes running 2021-06-04T02:40:08.346 INFO:teuthology.kill:Nuking machines: ['smithi164', 'smithi170'] 2021-06-04T02:40:09.409 INFO:teuthology.kill:2021-06-04 02:40:09,409.409 INFO:teuthology.nuke:targets: 2021-06-04T02:40:09.410 INFO:teuthology.kill: smithi164.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5B1ejR+SlKbKtHezwL7BrSBbjnLj/W+yfHGzqhD/aEVgjduL1PBIwcsV9tCAlKFyG98m3rvFqOIheG86gsZQIUtH6/bd3UhFTbxzTl4EMW/I8lsEglmtqs35+g11pamnLXCOlW+iYnGWJdk/PQ3zRyjMNIDxBII216rH99VhGOLM5ObJIkw1XXxXuGx9s7d5NEFcNpuF6VBgobcjQCVTKH/9qzGUfwxbyMfmHR4LfE9eu9Qccd4G7fCEMryLagHebtChI4CWTUXUs2GqybsKiKSkkelavw3WN7E2oqaVlzHDcV5bVH67VSpuEBKWL3pV2TKCV4d4bv5tHeCL0V9z1ttJn3BQMSdP6v9RDx5aoH+UOJTBPrNA+N7OEKG91avxxH5GYzhHG354idPrSwZKdplYg1UI9T2+NJMgdpvL/IrTOOEFmyVaRvMLe4PMt9gx1qbPo0fan1ayxeDn659RB5EouhNVs4EIX4m9x2MB3z6SsAxIanDpq/sSnYwNmDZE= 2021-06-04T02:40:09.410 INFO:teuthology.kill: smithi170.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1e+HSlbTazgxj8wVSjYsu31FjNaEyTEZ2514aiOTBUEcW2wbtBOh/yc6h9dbQbme4IAQ5yru9mNSk0Cr0kWo/L+UspvDjwTZM7zkmp2eTDqydYFLYsz8kggcAr9BUsE7tVZiQ00Zlz0Yzp6sXdQKC1yJZKU1tU4UccdBfOjrl0tRcwsXYoShNcN3eL2VXauy6M5Cwfg+5BGC6PBH49U9wwiahLWvCdcGwxLNZCWiQRQIL9iAFZUF0dHCuL9x98BaS/kryQMechDnGTgSMvQGbc8cS4zZ7Hr++bs37AxmI7K1rJZRo2tDd1X3b5ep1Pak8QiaHlSy398djZh5eOqIFHb0MhVXOMwvv91gZk0dQu0kfsQXSOB43tOyc8CshFxqBNUJlsY7VLAq8rrTgmBkBj5dMUW0LJgfs2LH82gaEFNvw+z3f0Oh7TEszRxa1jIOKYIkxvte5qenNqjgqR7xeS1Nv06tXtwAEAiIB9vsVwnC7mIeezxJg5gxDuBSoHq8= 2021-06-04T02:40:09.431 INFO:teuthology.kill:2021-06-04 02:40:09,431.431 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-04T02:40:09.434 INFO:teuthology.kill:2021-06-04 02:40:09,434.434 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-04T02:40:09.497 INFO:teuthology.kill:2021-06-04 02:40:09,497.497 INFO:teuthology.orchestra.console:Power off smithi164 2021-06-04T02:40:09.511 INFO:teuthology.kill:2021-06-04 02:40:09,510.510 INFO:teuthology.orchestra.console:Power off smithi170 2021-06-04T02:40:09.639 INFO:teuthology.kill:2021-06-04 02:40:09,639.639 INFO:teuthology.orchestra.console:Power off for smithi164 completed 2021-06-04T02:40:09.653 INFO:teuthology.kill:2021-06-04 02:40:09,652.652 INFO:teuthology.orchestra.console:Power off for smithi170 completed 2021-06-04T02:40:09.783 INFO:teuthology.kill:2021-06-04 02:40:09,782.782 INFO:teuthology.lock.ops:unlocked smithi164.front.sepia.ceph.com 2021-06-04T02:40:09.801 INFO:teuthology.kill:2021-06-04 02:40:09,801.801 INFO:teuthology.lock.ops:unlocked smithi170.front.sepia.ceph.com 2021-06-04T02:42:10.091 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-06-04T02:42:10.215 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-06-04T02:42:10.216 INFO:teuthology.nuke:Checking targets against current locks 2021-06-04T02:42:10.300 INFO:teuthology.nuke:Not nuking smithi164.front.sepia.ceph.com because description doesn't match 2021-06-04T02:42:10.300 INFO:teuthology.nuke:Not nuking smithi170.front.sepia.ceph.com because description doesn't match