2021-02-25T17:31:28.353 INFO:root:teuthology version: 1.1.0-9a6b849c 2021-02-25T17:31:28.451 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi086.front.sepia.ceph.com' reimaging 2021-02-25T17:31:28.452 INFO:teuthology.lock.ops:Updating [ubuntu@smithi086.front.sepia.ceph.com]: reset os type and version on server 2021-02-25T17:31:28.453 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2021-02-25T17:31:28.468 INFO:teuthology.lock.ops:Node 'ubuntu@smithi086.front.sepia.ceph.com' reimaging is complete 2021-02-25T17:31:28.468 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi187.front.sepia.ceph.com' reimaging 2021-02-25T17:31:28.469 INFO:teuthology.lock.ops:Updating [ubuntu@smithi187.front.sepia.ceph.com]: reset os type and version on server 2021-02-25T17:31:28.469 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2021-02-25T17:31:28.484 INFO:teuthology.lock.ops:Node 'ubuntu@smithi187.front.sepia.ceph.com' reimaging is complete 2021-02-25T17:31:28.660 INFO:teuthology.provision.fog.smithi086:Scheduling deploy of rhel 8.3 2021-02-25T17:31:28.670 INFO:teuthology.provision.fog.smithi187:Scheduling deploy of rhel 8.3 2021-02-25T17:31:28.969 INFO:teuthology.orchestra.console:Power off smithi086 2021-02-25T17:31:28.986 INFO:teuthology.orchestra.console:Power off smithi187 2021-02-25T17:31:29.121 INFO:teuthology.orchestra.console:Power off for smithi086 completed 2021-02-25T17:31:29.133 INFO:teuthology.orchestra.console:Power off for smithi187 completed 2021-02-25T17:31:29.223 INFO:teuthology.orchestra.console:Power on smithi086 2021-02-25T17:31:29.234 INFO:teuthology.orchestra.console:Power on smithi187 2021-02-25T17:31:33.493 INFO:teuthology.orchestra.console:Power on for smithi187 completed 2021-02-25T17:31:34.199 INFO:teuthology.provision.fog.smithi187:Waiting for deploy to finish 2021-02-25T17:31:37.598 INFO:teuthology.orchestra.console:Power on for smithi086 completed 2021-02-25T17:31:37.699 INFO:teuthology.provision.fog.smithi086:Waiting for deploy to finish 2021-02-25T17:36:06.099 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-25T17:36:06.786 INFO:teuthology.provision.fog.smithi187:Node is ready 2021-02-25T17:36:06.802 INFO:teuthology.orchestra.run.smithi187.stdout:smithi187.front.sepia.ceph.com 2021-02-25T17:36:06.858 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.15.187 smithi187.front.sepia.ceph.com smithi187 2021-02-25T17:36:07.180 INFO:teuthology.provision.fog.smithi187:Deploy complete! 2021-02-25T17:36:09.554 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-25T17:36:09.855 INFO:teuthology.provision.fog.smithi086:Node is ready 2021-02-25T17:36:09.869 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086.front.sepia.ceph.com 2021-02-25T17:36:09.924 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86 smithi086.front.sepia.ceph.com smithi086 2021-02-25T17:36:10.250 INFO:teuthology.provision.fog.smithi086:Deploy complete! 2021-02-25T17:36:10.427 INFO:teuthology.lock.ops:Checking smithi086.front.sepia.ceph.com 2021-02-25T17:36:10.432 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-25T17:36:10.452 INFO:teuthology.lock.ops:Checking smithi187.front.sepia.ceph.com 2021-02-25T17:36:10.452 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-25T17:36:10.468 INFO:teuthology.lock.ops:Updating [smithi086.front.sepia.ceph.com]: set os type and version on server 2021-02-25T17:36:10.470 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-25T17:36:10.803 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2021-02-25T17:36:23.041 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Red Hat Enterprise Linux" 2021-02-25T17:36:23.041 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="8.3 (Ootpa)" 2021-02-25T17:36:23.041 INFO:teuthology.orchestra.run.smithi086.stdout:ID="rhel" 2021-02-25T17:36:23.042 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="fedora" 2021-02-25T17:36:23.042 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="8.3" 2021-02-25T17:36:23.042 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el8" 2021-02-25T17:36:23.042 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-02-25T17:36:23.043 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2021-02-25T17:36:23.043 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-02-25T17:36:23.043 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.redhat.com/" 2021-02-25T17:36:23.043 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-02-25T17:36:23.043 INFO:teuthology.orchestra.run.smithi086.stdout: 2021-02-25T17:36:23.044 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-02-25T17:36:23.044 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-02-25T17:36:23.044 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-02-25T17:36:23.044 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-02-25T17:36:23.045 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2021-02-25T17:36:23.061 INFO:teuthology.lock.ops:Updating [smithi187.front.sepia.ceph.com]: set os type and version on server 2021-02-25T17:36:23.061 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-25T17:36:23.351 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2021-02-25T17:36:23.405 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="Red Hat Enterprise Linux" 2021-02-25T17:36:23.406 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="8.3 (Ootpa)" 2021-02-25T17:36:23.406 INFO:teuthology.orchestra.run.smithi187.stdout:ID="rhel" 2021-02-25T17:36:23.406 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="fedora" 2021-02-25T17:36:23.407 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="8.3" 2021-02-25T17:36:23.407 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el8" 2021-02-25T17:36:23.407 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-02-25T17:36:23.407 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2021-02-25T17:36:23.407 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-02-25T17:36:23.408 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.redhat.com/" 2021-02-25T17:36:23.408 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-02-25T17:36:23.408 INFO:teuthology.orchestra.run.smithi187.stdout: 2021-02-25T17:36:23.408 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-02-25T17:36:23.408 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-02-25T17:36:23.409 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-02-25T17:36:23.409 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-02-25T17:36:23.410 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2021-02-25T17:36:23.489 INFO:teuthology.dispatcher.supervisor:Running job 5901935 2021-02-25T17:36:23.489 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology -v --owner scheduled_xiubli@teuthology --archive /home/teuthworker/archive/xiubli-2021-02-22_04:56:06-fs:functional-wip-lxb-testing-20210221-1527-distro-basic-smithi/5901935 --name xiubli-2021-02-22_04:56:06-fs:functional-wip-lxb-testing-20210221-1527-distro-basic-smithi --description fs:functional/{begin.yaml clusters/1a3s-mds-4c-client.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} distro/{rhel_8.yaml} mount/fuse.yaml objectstore/bluestore-bitmap.yaml overrides/{frag_enable.yaml no_client_pidfile.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/client-readahead.yaml} -- /home/teuthworker/archive/xiubli-2021-02-22_04:56:06-fs:functional-wip-lxb-testing-20210221-1527-distro-basic-smithi/5901935/orig.config.yaml 2021-02-25T17:36:23.496 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/xiubli-2021-02-22_04:56:06-fs:functional-wip-lxb-testing-20210221-1527-distro-basic-smithi/5901935 2021-02-25T17:36:23.497 INFO:teuthology.dispatcher.supervisor:Job PID: 10455 2021-02-25T17:36:23.498 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-02-26T05:36:40.014 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-02-26T05:36:40.932 INFO:teuthology.kill:Killing Pids: {10455} 2021-02-26T05:36:42.055 INFO:teuthology.kill:Nuking machines: ['smithi086', 'smithi187'] 2021-02-26T05:36:42.857 INFO:teuthology.kill:2021-02-26 05:36:42,856.856 INFO:teuthology.nuke:targets: 2021-02-26T05:36:42.857 INFO:teuthology.kill: smithi086.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCKanwO/4OOibQxBptwuAWt/jZ7IrFdlySCW7EOdQKV7FipsHEgzkw8pXgDaT7yHamyPDias+6OCbz2H8b2tjHicLXKMQLeuU+FL6JNgclNIWxsc1zZcqIsXJwWhlU8Vnebc/QwmWqxyfcgW/aFSJMPEx38CcYsxRQTcYBQFYuVY1RDUMBcWoDOJ5uxQnZzQieFLb4vZ4t0xeosRjbTbIhGQlV53YM5EVt9dR50/3TgV2h79iXY81ZJxOww9en6dhertAkyVjOe5bmq4qzBxP/79Mg0Q5AV1U6/JqBujtSntUV2uESQZ0SfjhX5C7WLb1Js+D4kXcR//+/Mdt+mUp5sJ0KLYIf7NGuBXXqp7DSMz1odBMfCwBBH9RPO4oUOj2CyT9v0bQtx5urBYZ77bkIhRMwQbVueG7E9ALIHYw5ursC5kXNU2qDCZ/LMyh0YOQpJn/D+QkqtmxHZKYJdNHzI0/CRiNmf6jWyYZCIr8l499YVuMjpwotan3/b/mGtVU= 2021-02-26T05:36:42.858 INFO:teuthology.kill: smithi187.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPkX0V9CgPLrYGdWc2V3gHXqHOoEJBtD7UauFMeIzJXKEzuS2649mx7eBnNQZbZgiLEXGAu0lqE4mlF+FvShszkDMOlQXog3/RinIbrUao5+kWXPchoM683/ZMP4oqD+pKHWkYiLq2IYBFpwAvSK4wwtnRyMzWDL6VAMnovtN+SZcF+2bfXN3B2GtembcdHXX/cNyAygsWaZQzMZoLLC4Fcf7IuMPXyVSXrqwbBKksyLZ/auCKSMz8t8OzFfYAHPfl8I8NqCwgTtrXkXWZjoT9wz9+nY8IVHXS/hQYBM+mgWPWtDcwKLTjGBnhSFf27JHcxDJGpCucLjGsKIYDA3Ne/EtvGDltumXQ4/orPi+j37Dt59xB3EgviCjUm+3b8mEY2aMPjaiqbLzfWc4F7aK3ajdUVXW3UkQcPLCMju//kStDei6E3Akr5UyFPQ8v3pm/bj+gml6qbCj3p7bSTp5HhLX/P5I3A3h4CEjrInxrr/O5khDAcklib6hke+IPSKs= 2021-02-26T05:36:42.859 INFO:teuthology.kill:2021-02-26 05:36:42,857.857 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-26T05:36:42.863 INFO:teuthology.kill:2021-02-26 05:36:42,863.863 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-26T05:36:42.926 INFO:teuthology.kill:2021-02-26 05:36:42,925.925 INFO:teuthology.orchestra.console:Power off smithi187 2021-02-26T05:36:42.938 INFO:teuthology.kill:2021-02-26 05:36:42,938.938 INFO:teuthology.orchestra.console:Power off smithi086 2021-02-26T05:36:51.293 INFO:teuthology.kill:2021-02-26 05:36:51,293.293 INFO:teuthology.orchestra.console:Power off for smithi187 completed 2021-02-26T05:36:52.418 INFO:teuthology.kill:2021-02-26 05:36:51,299.299 INFO:teuthology.orchestra.console:Power off for smithi086 completed 2021-02-26T05:36:52.495 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2021-02-26T05:36:52.497 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2021-02-26T05:36:52.500 INFO:teuthology.misc:Compressing logs... 2021-02-26T05:36:52.510 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-26T05:37:27.248 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/misc.py", line 1374, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-02-26T05:37:27.322 INFO:teuthology.kill:No teuthology processes running 2021-02-26T05:37:28.219 INFO:teuthology.kill:Nuking machines: ['smithi086', 'smithi187'] 2021-02-26T05:37:28.985 INFO:teuthology.kill:2021-02-26 05:37:28,984.984 INFO:teuthology.nuke:targets: 2021-02-26T05:37:28.986 INFO:teuthology.kill: smithi086.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCKanwO/4OOibQxBptwuAWt/jZ7IrFdlySCW7EOdQKV7FipsHEgzkw8pXgDaT7yHamyPDias+6OCbz2H8b2tjHicLXKMQLeuU+FL6JNgclNIWxsc1zZcqIsXJwWhlU8Vnebc/QwmWqxyfcgW/aFSJMPEx38CcYsxRQTcYBQFYuVY1RDUMBcWoDOJ5uxQnZzQieFLb4vZ4t0xeosRjbTbIhGQlV53YM5EVt9dR50/3TgV2h79iXY81ZJxOww9en6dhertAkyVjOe5bmq4qzBxP/79Mg0Q5AV1U6/JqBujtSntUV2uESQZ0SfjhX5C7WLb1Js+D4kXcR//+/Mdt+mUp5sJ0KLYIf7NGuBXXqp7DSMz1odBMfCwBBH9RPO4oUOj2CyT9v0bQtx5urBYZ77bkIhRMwQbVueG7E9ALIHYw5ursC5kXNU2qDCZ/LMyh0YOQpJn/D+QkqtmxHZKYJdNHzI0/CRiNmf6jWyYZCIr8l499YVuMjpwotan3/b/mGtVU= 2021-02-26T05:37:28.986 INFO:teuthology.kill: smithi187.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPkX0V9CgPLrYGdWc2V3gHXqHOoEJBtD7UauFMeIzJXKEzuS2649mx7eBnNQZbZgiLEXGAu0lqE4mlF+FvShszkDMOlQXog3/RinIbrUao5+kWXPchoM683/ZMP4oqD+pKHWkYiLq2IYBFpwAvSK4wwtnRyMzWDL6VAMnovtN+SZcF+2bfXN3B2GtembcdHXX/cNyAygsWaZQzMZoLLC4Fcf7IuMPXyVSXrqwbBKksyLZ/auCKSMz8t8OzFfYAHPfl8I8NqCwgTtrXkXWZjoT9wz9+nY8IVHXS/hQYBM+mgWPWtDcwKLTjGBnhSFf27JHcxDJGpCucLjGsKIYDA3Ne/EtvGDltumXQ4/orPi+j37Dt59xB3EgviCjUm+3b8mEY2aMPjaiqbLzfWc4F7aK3ajdUVXW3UkQcPLCMju//kStDei6E3Akr5UyFPQ8v3pm/bj+gml6qbCj3p7bSTp5HhLX/P5I3A3h4CEjrInxrr/O5khDAcklib6hke+IPSKs= 2021-02-26T05:37:29.008 INFO:teuthology.kill:2021-02-26 05:37:29,007.007 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-26T05:37:29.010 INFO:teuthology.kill:2021-02-26 05:37:29,010.010 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-26T05:37:29.067 INFO:teuthology.kill:2021-02-26 05:37:29,067.067 INFO:teuthology.orchestra.console:Power off smithi187 2021-02-26T05:37:29.079 INFO:teuthology.kill:2021-02-26 05:37:29,078.078 INFO:teuthology.orchestra.console:Power off smithi086 2021-02-26T05:37:29.205 INFO:teuthology.kill:2021-02-26 05:37:29,204.204 INFO:teuthology.orchestra.console:Power off for smithi187 completed 2021-02-26T05:37:29.218 INFO:teuthology.kill:2021-02-26 05:37:29,218.218 INFO:teuthology.orchestra.console:Power off for smithi086 completed 2021-02-26T05:37:29.351 INFO:teuthology.kill:2021-02-26 05:37:29,350.350 INFO:teuthology.lock.ops:unlocked smithi187.front.sepia.ceph.com 2021-02-26T05:37:29.361 INFO:teuthology.kill:2021-02-26 05:37:29,361.361 INFO:teuthology.lock.ops:unlocked smithi086.front.sepia.ceph.com 2021-02-26T05:39:29.581 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-02-26T05:39:29.726 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-02-26T05:39:29.729 INFO:teuthology.nuke:Checking targets against current locks 2021-02-26T05:39:30.034 INFO:teuthology.nuke:Not nuking smithi187.front.sepia.ceph.com because description doesn't match 2021-02-26T05:39:30.034 INFO:teuthology.nuke:Not nuking smithi086.front.sepia.ceph.com because description doesn't match