2021-08-23T22:37:15.380 INFO:root:teuthology version: 1.1.0-321319b1 2021-08-23T22:37:15.496 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi041.front.sepia.ceph.com' reimaging 2021-08-23T22:37:15.498 INFO:teuthology.lock.ops:Updating [ubuntu@smithi041.front.sepia.ceph.com]: reset os type and version on server 2021-08-23T22:37:15.498 INFO:teuthology.lock.ops:Updating smithi041.front.sepia.ceph.com on lock server 2021-08-23T22:37:15.516 INFO:teuthology.lock.ops:Node 'ubuntu@smithi041.front.sepia.ceph.com' reimaging is complete 2021-08-23T22:37:15.516 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi142.front.sepia.ceph.com' reimaging 2021-08-23T22:37:15.517 INFO:teuthology.lock.ops:Updating [ubuntu@smithi142.front.sepia.ceph.com]: reset os type and version on server 2021-08-23T22:37:15.517 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2021-08-23T22:37:15.535 INFO:teuthology.lock.ops:Node 'ubuntu@smithi142.front.sepia.ceph.com' reimaging is complete 2021-08-23T22:37:15.722 INFO:teuthology.provision.fog.smithi041:Scheduling deploy of ubuntu 20.04 2021-08-23T22:37:15.726 INFO:teuthology.provision.fog.smithi142:Scheduling deploy of ubuntu 20.04 2021-08-23T22:37:16.011 INFO:teuthology.orchestra.console:Power off smithi142 2021-08-23T22:37:16.031 INFO:teuthology.orchestra.console:Power off smithi041 2021-08-23T22:37:24.391 INFO:teuthology.orchestra.console:Power off for smithi142 completed 2021-08-23T22:37:24.409 INFO:teuthology.orchestra.console:Power off for smithi041 completed 2021-08-23T22:37:24.494 INFO:teuthology.orchestra.console:Power on smithi142 2021-08-23T22:37:24.510 INFO:teuthology.orchestra.console:Power on smithi041 2021-08-23T22:37:32.871 INFO:teuthology.orchestra.console:Power on for smithi142 completed 2021-08-23T22:37:32.879 INFO:teuthology.orchestra.console:Power on for smithi041 completed 2021-08-23T22:37:32.974 INFO:teuthology.provision.fog.smithi142:Waiting for deploy to finish 2021-08-23T22:37:32.980 INFO:teuthology.provision.fog.smithi041:Waiting for deploy to finish 2021-08-23T22:42:10.762 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-08-23T22:42:11.264 INFO:teuthology.provision.fog.smithi041:Node is ready 2021-08-23T22:42:11.314 INFO:teuthology.orchestra.run.smithi041.stdout:smithi041.front.sepia.ceph.com 2021-08-23T22:42:11.322 INFO:teuthology.orchestra.run.smithi041.stdout:172.21.15.41 smithi041.front.sepia.ceph.com smithi041 2021-08-23T22:42:11.478 INFO:teuthology.provision.fog.smithi041:Deploy complete! 2021-08-23T22:42:31.252 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-08-23T22:42:31.756 INFO:teuthology.provision.fog.smithi142:Node is ready 2021-08-23T22:42:31.763 INFO:teuthology.orchestra.run.smithi142.stdout:smithi142.front.sepia.ceph.com 2021-08-23T22:42:31.771 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.142 smithi142.front.sepia.ceph.com smithi142 2021-08-23T22:42:31.979 INFO:teuthology.provision.fog.smithi142:Deploy complete! 2021-08-23T22:42:32.142 INFO:teuthology.lock.ops:Checking smithi041.front.sepia.ceph.com 2021-08-23T22:42:32.143 INFO:teuthology.lock.ops:New key found. Updating... 2021-08-23T22:42:32.167 INFO:teuthology.lock.ops:Checking smithi142.front.sepia.ceph.com 2021-08-23T22:42:32.168 INFO:teuthology.lock.ops:New key found. Updating... 2021-08-23T22:42:32.184 INFO:teuthology.lock.ops:Updating [smithi041.front.sepia.ceph.com]: set os type and version on server 2021-08-23T22:42:32.185 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-23T22:42:32.443 INFO:teuthology.orchestra.run.smithi041.stdout:x86_64 2021-08-23T22:42:32.449 INFO:teuthology.orchestra.run.smithi041.stdout:NAME="Ubuntu" 2021-08-23T22:42:32.450 INFO:teuthology.orchestra.run.smithi041.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-08-23T22:42:32.450 INFO:teuthology.orchestra.run.smithi041.stdout:ID=ubuntu 2021-08-23T22:42:32.450 INFO:teuthology.orchestra.run.smithi041.stdout:ID_LIKE=debian 2021-08-23T22:42:32.451 INFO:teuthology.orchestra.run.smithi041.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-08-23T22:42:32.451 INFO:teuthology.orchestra.run.smithi041.stdout:VERSION_ID="20.04" 2021-08-23T22:42:32.452 INFO:teuthology.orchestra.run.smithi041.stdout:HOME_URL="https://www.ubuntu.com/" 2021-08-23T22:42:32.452 INFO:teuthology.orchestra.run.smithi041.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-08-23T22:42:32.452 INFO:teuthology.orchestra.run.smithi041.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-08-23T22:42:32.453 INFO:teuthology.orchestra.run.smithi041.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-08-23T22:42:32.453 INFO:teuthology.orchestra.run.smithi041.stdout:VERSION_CODENAME=focal 2021-08-23T22:42:32.453 INFO:teuthology.orchestra.run.smithi041.stdout:UBUNTU_CODENAME=focal 2021-08-23T22:42:32.455 INFO:teuthology.lock.ops:Updating smithi041.front.sepia.ceph.com on lock server 2021-08-23T22:42:32.474 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: set os type and version on server 2021-08-23T22:42:32.475 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-23T22:42:32.638 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2021-08-23T22:42:32.644 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="Ubuntu" 2021-08-23T22:42:32.645 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-08-23T22:42:32.645 INFO:teuthology.orchestra.run.smithi142.stdout:ID=ubuntu 2021-08-23T22:42:32.645 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE=debian 2021-08-23T22:42:32.645 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-08-23T22:42:32.646 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="20.04" 2021-08-23T22:42:32.646 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://www.ubuntu.com/" 2021-08-23T22:42:32.646 INFO:teuthology.orchestra.run.smithi142.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-08-23T22:42:32.646 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-08-23T22:42:32.647 INFO:teuthology.orchestra.run.smithi142.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-08-23T22:42:32.647 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_CODENAME=focal 2021-08-23T22:42:32.647 INFO:teuthology.orchestra.run.smithi142.stdout:UBUNTU_CODENAME=focal 2021-08-23T22:42:32.648 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2021-08-23T22:42:32.728 INFO:teuthology.dispatcher.supervisor:Running job 6353758 2021-08-23T22:42:32.729 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-08-23_19:25:10-fs-wip-yuri3-testing-2021-08-23-0808-distro-basic-smithi/6353758 --name yuriw-2021-08-23_19:25:10-fs-wip-yuri3-testing-2021-08-23-0808-distro-basic-smithi --description fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/fragment} -- /home/teuthworker/archive/yuriw-2021-08-23_19:25:10-fs-wip-yuri3-testing-2021-08-23-0808-distro-basic-smithi/6353758/orig.config.yaml 2021-08-23T22:42:32.733 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-08-23_19:25:10-fs-wip-yuri3-testing-2021-08-23-0808-distro-basic-smithi/6353758 2021-08-23T22:42:32.734 INFO:teuthology.dispatcher.supervisor:Job PID: 11868 2021-08-23T22:42:32.734 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-08-24T10:43:24.884 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-08-24T10:43:25.089 INFO:teuthology.kill:Killing Pids: {11868} 2021-08-24T10:43:26.275 INFO:teuthology.kill:Nuking machines: ['smithi041', 'smithi142'] 2021-08-24T10:43:27.276 INFO:teuthology.kill:2021-08-24 10:43:27,276.276 INFO:teuthology.nuke:targets: 2021-08-24T10:43:27.277 INFO:teuthology.kill: smithi041.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfcZ0NeHqdQNxB0goOr4wXKdn9jhMwVDlco4P+vvr5ss8Qiot8o7IDhL0JvDDGwr2ibpjHscgcAgZMlalJ0YBTsiiV4noN66DbEOQwe1n/32LEjc8aZPTcAiZADoOqk1fh1awkB4hIfiTQcGRNqd0CUdG5Qh6X/JgE5uRNw06kq4uFuoYD2jxJpR8UkAKJqbl5DFu0Rh7FOV024HaOjGcLcJGZRvGIgzzWtUT4DkqsaWo92i5ONYdQwKTF7L0a4Gs5YNhWbK5uhUG/tZIC1Iqjcu+WIx4alpYZ4unnepTj/eyXWdC9TE4WQ3qDX2OjuloUGzhBV4+pmN1VaSunRtl0t69CVN8eaZmzgVTd7u4uKBVBbbAwLThsAgrsOq9TZewugnJn99ICPU9OBiRu8Cptjj8dIofmC/0Dc7FCv/O+I8GkugRAMh6jA85MoN4rROrob0qIknk5DCRmVKEN413bwOp3R3SgcS7J2l3o11cTjKXvhdmlkGU55TXOeYZq+Ss= 2021-08-24T10:43:27.277 INFO:teuthology.kill: smithi142.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRlKwC35SYFLXqJzbmOU08BgyO07GKyNq8Rlcm6qAPP+/yvpi3cGI/a0kZQL6DAWUZsbEOk7joqtPIH6PacVcanhF24nd6iUzOdGQgpMMmTmRZM8vqcdzxzaRKvYWxmXHSGIAltxJt6wtYC0ggQpeGRFHChd6q2Iq1oVt2BZW2jr1hdODVRA8EbIp8JxaYe+D7+dsXMDmkmfXlM1GZ9186CqN/vdYZczSeJdKC0JZim2gJJJvyBmSKESJpQft/YBPsuWmDXcpMAuQh245wpuCbp24dNsBPZ1dIvRXC5gduGz/3anGv9Jmf+4PeesP+ICumjVFo2binBeK/xicxz1HmSlRCHQFDYwyOrVE2aazKCaYiLms1ZiDA1ogb9MwXmeJEzEzb8rioc3LTualq6ncA+vMa6f3WBAW13nr2OaTfKqgXGgnRY7qvKCu9QCWs6OQpEuDgZ57L/j3C+uux6YqzdONk0HBUTRwGhXDGRdEosvJhjYR9kt8Ghvq3Z9Dveok= 2021-08-24T10:43:27.278 INFO:teuthology.kill:2021-08-24 10:43:27,276.276 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-24T10:43:27.279 INFO:teuthology.kill:2021-08-24 10:43:27,279.279 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-24T10:43:27.344 INFO:teuthology.kill:2021-08-24 10:43:27,343.343 INFO:teuthology.orchestra.console:Power off smithi041 2021-08-24T10:43:27.356 INFO:teuthology.kill:2021-08-24 10:43:27,356.356 INFO:teuthology.orchestra.console:Power off smithi142 2021-08-24T10:43:35.713 INFO:teuthology.kill:2021-08-24 10:43:35,713.713 INFO:teuthology.orchestra.console:Power off for smithi041 completed 2021-08-24T10:43:39.838 INFO:teuthology.kill:2021-08-24 10:43:39,837.837 INFO:teuthology.orchestra.console:Power off for smithi142 completed 2021-08-24T10:43:40.140 INFO:teuthology.task.internal:roles: ubuntu@smithi041.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2021-08-24T10:43:40.140 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2021-08-24T10:43:40.141 INFO:teuthology.misc:Compressing logs... 2021-08-24T10:43:40.141 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-24T10:44:40.203 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-08-24T10:44:40.281 INFO:teuthology.kill:No teuthology processes running 2021-08-24T10:44:41.504 INFO:teuthology.kill:Nuking machines: ['smithi041', 'smithi142'] 2021-08-24T10:44:42.491 INFO:teuthology.kill:2021-08-24 10:44:42,491.491 INFO:teuthology.nuke:targets: 2021-08-24T10:44:42.492 INFO:teuthology.kill: smithi041.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfcZ0NeHqdQNxB0goOr4wXKdn9jhMwVDlco4P+vvr5ss8Qiot8o7IDhL0JvDDGwr2ibpjHscgcAgZMlalJ0YBTsiiV4noN66DbEOQwe1n/32LEjc8aZPTcAiZADoOqk1fh1awkB4hIfiTQcGRNqd0CUdG5Qh6X/JgE5uRNw06kq4uFuoYD2jxJpR8UkAKJqbl5DFu0Rh7FOV024HaOjGcLcJGZRvGIgzzWtUT4DkqsaWo92i5ONYdQwKTF7L0a4Gs5YNhWbK5uhUG/tZIC1Iqjcu+WIx4alpYZ4unnepTj/eyXWdC9TE4WQ3qDX2OjuloUGzhBV4+pmN1VaSunRtl0t69CVN8eaZmzgVTd7u4uKBVBbbAwLThsAgrsOq9TZewugnJn99ICPU9OBiRu8Cptjj8dIofmC/0Dc7FCv/O+I8GkugRAMh6jA85MoN4rROrob0qIknk5DCRmVKEN413bwOp3R3SgcS7J2l3o11cTjKXvhdmlkGU55TXOeYZq+Ss= 2021-08-24T10:44:42.493 INFO:teuthology.kill: smithi142.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRlKwC35SYFLXqJzbmOU08BgyO07GKyNq8Rlcm6qAPP+/yvpi3cGI/a0kZQL6DAWUZsbEOk7joqtPIH6PacVcanhF24nd6iUzOdGQgpMMmTmRZM8vqcdzxzaRKvYWxmXHSGIAltxJt6wtYC0ggQpeGRFHChd6q2Iq1oVt2BZW2jr1hdODVRA8EbIp8JxaYe+D7+dsXMDmkmfXlM1GZ9186CqN/vdYZczSeJdKC0JZim2gJJJvyBmSKESJpQft/YBPsuWmDXcpMAuQh245wpuCbp24dNsBPZ1dIvRXC5gduGz/3anGv9Jmf+4PeesP+ICumjVFo2binBeK/xicxz1HmSlRCHQFDYwyOrVE2aazKCaYiLms1ZiDA1ogb9MwXmeJEzEzb8rioc3LTualq6ncA+vMa6f3WBAW13nr2OaTfKqgXGgnRY7qvKCu9QCWs6OQpEuDgZ57L/j3C+uux6YqzdONk0HBUTRwGhXDGRdEosvJhjYR9kt8Ghvq3Z9Dveok= 2021-08-24T10:44:42.513 INFO:teuthology.kill:2021-08-24 10:44:42,513.513 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-24T10:44:42.518 INFO:teuthology.kill:2021-08-24 10:44:42,518.518 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-24T10:44:42.582 INFO:teuthology.kill:2021-08-24 10:44:42,582.582 INFO:teuthology.orchestra.console:Power off smithi041 2021-08-24T10:44:42.595 INFO:teuthology.kill:2021-08-24 10:44:42,595.595 INFO:teuthology.orchestra.console:Power off smithi142 2021-08-24T10:44:42.724 INFO:teuthology.kill:2021-08-24 10:44:42,723.723 INFO:teuthology.orchestra.console:Power off for smithi041 completed 2021-08-24T10:44:42.737 INFO:teuthology.kill:2021-08-24 10:44:42,736.736 INFO:teuthology.orchestra.console:Power off for smithi142 completed 2021-08-24T10:44:42.870 INFO:teuthology.kill:2021-08-24 10:44:42,869.869 INFO:teuthology.lock.ops:unlocked smithi041.front.sepia.ceph.com 2021-08-24T10:44:42.888 INFO:teuthology.kill:2021-08-24 10:44:42,888.888 INFO:teuthology.lock.ops:unlocked smithi142.front.sepia.ceph.com 2021-08-24T10:46:43.239 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-08-24T10:46:43.337 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-08-24T10:46:43.338 INFO:teuthology.nuke:Checking targets against current locks 2021-08-24T10:46:43.455 INFO:teuthology.nuke:Not nuking smithi142.front.sepia.ceph.com because description doesn't match 2021-08-24T10:46:43.455 INFO:teuthology.nuke:Not nuking smithi041.front.sepia.ceph.com because description doesn't match