2021-02-06T03:20:35.621 INFO:root:teuthology version: 1.1.0-17c09965 2021-02-06T03:20:35.728 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi169.front.sepia.ceph.com' reimaging 2021-02-06T03:20:35.729 INFO:teuthology.lock.ops:Updating [ubuntu@smithi169.front.sepia.ceph.com]: reset os type and version on server 2021-02-06T03:20:35.729 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2021-02-06T03:20:35.743 INFO:teuthology.lock.ops:Node 'ubuntu@smithi169.front.sepia.ceph.com' reimaging is complete 2021-02-06T03:20:35.744 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi204.front.sepia.ceph.com' reimaging 2021-02-06T03:20:35.744 INFO:teuthology.lock.ops:Updating [ubuntu@smithi204.front.sepia.ceph.com]: reset os type and version on server 2021-02-06T03:20:35.745 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2021-02-06T03:20:35.759 INFO:teuthology.lock.ops:Node 'ubuntu@smithi204.front.sepia.ceph.com' reimaging is complete 2021-02-06T03:20:35.947 INFO:teuthology.provision.fog.smithi204:Scheduling deploy of ubuntu 16.04 2021-02-06T03:20:35.956 INFO:teuthology.provision.fog.smithi169:Scheduling deploy of ubuntu 16.04 2021-02-06T03:20:36.277 INFO:teuthology.orchestra.console:Power off smithi204 2021-02-06T03:20:36.298 INFO:teuthology.orchestra.console:Power off smithi169 2021-02-06T03:20:44.681 INFO:teuthology.orchestra.console:Power off for smithi169 completed 2021-02-06T03:20:44.782 INFO:teuthology.orchestra.console:Power on smithi169 2021-02-06T03:20:48.796 INFO:teuthology.orchestra.console:Power off for smithi204 completed 2021-02-06T03:20:48.899 INFO:teuthology.orchestra.console:Power on smithi204 2021-02-06T03:20:53.141 INFO:teuthology.orchestra.console:Power on for smithi169 completed 2021-02-06T03:20:53.243 INFO:teuthology.provision.fog.smithi169:Waiting for deploy to finish 2021-02-06T03:20:57.259 INFO:teuthology.orchestra.console:Power on for smithi204 completed 2021-02-06T03:20:57.360 INFO:teuthology.provision.fog.smithi204:Waiting for deploy to finish 2021-02-06T03:25:33.388 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-02-06T03:25:36.350 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-06T03:25:36.716 INFO:teuthology.provision.fog.smithi169:Node is ready 2021-02-06T03:25:36.756 INFO:teuthology.orchestra.run.smithi169.stdout:smithi169.front.sepia.ceph.com 2021-02-06T03:25:36.797 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.15.169 smithi169.front.sepia.ceph.com smithi169 2021-02-06T03:25:37.056 INFO:teuthology.provision.fog.smithi169:Deploy complete! 2021-02-06T03:25:42.224 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-02-06T03:25:48.235 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-02-06T03:25:54.423 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-06T03:25:54.770 INFO:teuthology.provision.fog.smithi204:Node is ready 2021-02-06T03:25:54.822 INFO:teuthology.orchestra.run.smithi204.stdout:smithi204.front.sepia.ceph.com 2021-02-06T03:25:54.870 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.15.204 smithi204.front.sepia.ceph.com smithi204 2021-02-06T03:25:55.184 INFO:teuthology.provision.fog.smithi204:Deploy complete! 2021-02-06T03:25:55.402 INFO:teuthology.lock.ops:Checking smithi169.front.sepia.ceph.com 2021-02-06T03:25:55.403 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-06T03:25:55.420 INFO:teuthology.lock.ops:Checking smithi204.front.sepia.ceph.com 2021-02-06T03:25:55.421 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-06T03:25:55.434 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: set os type and version on server 2021-02-06T03:25:55.434 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-06T03:25:55.744 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2021-02-06T03:25:55.784 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="Ubuntu" 2021-02-06T03:25:55.785 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="16.04.6 LTS (Xenial Xerus)" 2021-02-06T03:25:55.785 INFO:teuthology.orchestra.run.smithi169.stdout:ID=ubuntu 2021-02-06T03:25:55.785 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE=debian 2021-02-06T03:25:55.785 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="Ubuntu 16.04.6 LTS" 2021-02-06T03:25:55.786 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="16.04" 2021-02-06T03:25:55.786 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="http://www.ubuntu.com/" 2021-02-06T03:25:55.786 INFO:teuthology.orchestra.run.smithi169.stdout:SUPPORT_URL="http://help.ubuntu.com/" 2021-02-06T03:25:55.786 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/" 2021-02-06T03:25:55.787 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_CODENAME=xenial 2021-02-06T03:25:55.787 INFO:teuthology.orchestra.run.smithi169.stdout:UBUNTU_CODENAME=xenial 2021-02-06T03:25:55.788 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2021-02-06T03:25:55.804 INFO:teuthology.lock.ops:Updating [smithi204.front.sepia.ceph.com]: set os type and version on server 2021-02-06T03:25:55.804 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-06T03:25:56.092 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2021-02-06T03:25:56.132 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="Ubuntu" 2021-02-06T03:25:56.132 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="16.04.6 LTS (Xenial Xerus)" 2021-02-06T03:25:56.132 INFO:teuthology.orchestra.run.smithi204.stdout:ID=ubuntu 2021-02-06T03:25:56.133 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE=debian 2021-02-06T03:25:56.133 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="Ubuntu 16.04.6 LTS" 2021-02-06T03:25:56.133 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="16.04" 2021-02-06T03:25:56.133 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="http://www.ubuntu.com/" 2021-02-06T03:25:56.133 INFO:teuthology.orchestra.run.smithi204.stdout:SUPPORT_URL="http://help.ubuntu.com/" 2021-02-06T03:25:56.134 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/" 2021-02-06T03:25:56.134 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_CODENAME=xenial 2021-02-06T03:25:56.134 INFO:teuthology.orchestra.run.smithi204.stdout:UBUNTU_CODENAME=xenial 2021-02-06T03:25:56.134 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2021-02-06T03:25:56.213 INFO:teuthology.dispatcher.supervisor:Running job 5860347 2021-02-06T03:25:56.213 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/virtualenv/bin/teuthology -v --owner scheduled_khiremat@teuthology --archive /home/teuthworker/archive/khiremat-2021-02-06_03:12:31-fs-wip-khiremat-mgr-authid-bp-5-nautilus-distro-basic-smithi/5860347 --name khiremat-2021-02-06_03:12:31-fs-wip-khiremat-mgr-authid-bp-5-nautilus-distro-basic-smithi --description fs/basic_functional/{begin clusters/1-mds-4-client-coloc conf/{client mds mon osd} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} supported-random-distros$/{ubuntu_16.04} tasks/volumes} -- /home/teuthworker/archive/khiremat-2021-02-06_03:12:31-fs-wip-khiremat-mgr-authid-bp-5-nautilus-distro-basic-smithi/5860347/orig.config.yaml 2021-02-06T03:25:56.216 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/khiremat-2021-02-06_03:12:31-fs-wip-khiremat-mgr-authid-bp-5-nautilus-distro-basic-smithi/5860347 2021-02-06T03:25:56.217 INFO:teuthology.dispatcher.supervisor:Job PID: 8405 2021-02-06T03:25:56.217 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-02-06T15:26:12.298 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-02-06T15:26:12.416 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.1'] 2021-02-06T15:26:12.421 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.2', 'client.3'] 2021-02-06T15:26:12.421 INFO:teuthology.misc:Compressing logs... 2021-02-06T15:26:12.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-06T15:26:12.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-06T15:27:13.452 INFO:teuthology.kill:Killing Pids: {8405} 2021-02-06T15:27:15.197 INFO:teuthology.kill:Nuking machines: ['smithi169', 'smithi204'] 2021-02-06T15:27:17.807 INFO:teuthology.kill:2021-02-06 15:27:17,807.807 INFO:teuthology.nuke:targets: 2021-02-06T15:27:17.810 INFO:teuthology.kill: smithi169.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgQd91aDOVMptmSwyXy5KtkFYWTrQSJsGuZcqd0IBKUI4WXKB4x4q0YAZoPdYzr95Hhd7smWg5VmlUQa83dA56GeyS2YoelE+/e48NF8tdB3dJB86KR1nXvwddJneXo0SW2o5jUjytwoUligHK6A8w8kekjSH4OysOq91k6cNTEFxYgK/vcXWmuIPw+mBw2dMqDqHq9S08Hpu6Qf1YRFzrp7O2QQRezMmEua+9bQEGYHTQn2ttZLhfBdwjb0wK6h6NB1Dr78Dt1MTu9FGUjnWzXXSNQpU9+EW99cHGg5GkUJOXq3MpCd+spwLl2d5/DC9+1BXrjYSNw1ePB+tpLMJ/ 2021-02-06T15:27:17.810 INFO:teuthology.kill: smithi204.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDGGuV8DGn2Jge4RQ3eHZO88DEcZeE/YwpYv9VonZsk8/TLqddohmp8moN7MSac6J2Yoeg9dWSPXfvCxLb1O6rCOvEqQgqF2D+yBfV0bzYJV2p2Hc3L7N9Qy5eG1J0wPi4UB89KgZOp4nQEAJPKk3x5S9FQQ7R/q9RCmQjH9NyOqaiDrV88pvjDkvQxl3rrUZd9jf1cRSyYK0K7DMKe3k+j0bf1SMoCMINNO6fHbKWPWVPqGmXIIIx/Rtu00zujlMCLc/XgZN3LSv7TgEAWZxqm/nbvpTxt1QSARtLTxbbVdJaN0qqLHiQMR4uU7S4G0mei1GjCCCz6i4qjC6nChlxN 2021-02-06T15:27:17.845 INFO:teuthology.kill:2021-02-06 15:27:17,844.844 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-06T15:27:17.849 INFO:teuthology.kill:2021-02-06 15:27:17,849.849 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-06T15:27:17.923 INFO:teuthology.kill:2021-02-06 15:27:17,922.922 INFO:teuthology.orchestra.console:Power off smithi204 2021-02-06T15:27:17.948 INFO:teuthology.kill:2021-02-06 15:27:17,947.947 INFO:teuthology.orchestra.console:Power off smithi169 2021-02-06T15:27:26.355 INFO:teuthology.kill:2021-02-06 15:27:26,354.354 INFO:teuthology.orchestra.console:Power off for smithi204 completed 2021-02-06T15:27:26.503 INFO:teuthology.kill:2021-02-06 15:27:26,503.503 INFO:teuthology.lock.ops:unlocked smithi204.front.sepia.ceph.com 2021-02-06T15:27:30.471 INFO:teuthology.kill:2021-02-06 15:27:30,470.470 INFO:teuthology.orchestra.console:Power off for smithi169 completed 2021-02-06T15:27:30.613 INFO:teuthology.kill:2021-02-06 15:27:30,613.613 INFO:teuthology.lock.ops:unlocked smithi169.front.sepia.ceph.com 2021-02-06T15:29:30.800 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-02-06T15:29:30.923 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-02-06T15:29:30.923 INFO:teuthology.nuke:Checking targets against current locks 2021-02-06T15:29:31.014 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/teuthology/dispatcher/supervisor.py", line 59, in main verbose File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/teuthology/dispatcher/supervisor.py", line 152, in run_job unlock_targets(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/teuthology/dispatcher/supervisor.py", line 196, in unlock_targets nuke(fake_ctx, True) File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/teuthology/nuke/__init__.py", line 250, in nuke if ctx.name not in lock['description']: TypeError: argument of type 'NoneType' is not iterable