2021-03-09T07:10:12.455 INFO:root:teuthology version: 1.1.0-3082387b 2021-03-09T07:10:12.560 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba018.front.sepia.ceph.com' reimaging 2021-03-09T07:10:12.562 INFO:teuthology.lock.ops:Updating [ubuntu@gibba018.front.sepia.ceph.com]: reset os type and version on server 2021-03-09T07:10:12.563 INFO:teuthology.lock.ops:Updating gibba018.front.sepia.ceph.com on lock server 2021-03-09T07:10:12.583 INFO:teuthology.lock.ops:Node 'ubuntu@gibba018.front.sepia.ceph.com' reimaging is complete 2021-03-09T07:10:12.806 INFO:teuthology.provision.fog.gibba018:Scheduling deploy of ubuntu 18.04 2021-03-09T07:10:13.161 INFO:teuthology.orchestra.console:Power off gibba018 2021-03-09T07:10:21.618 INFO:teuthology.orchestra.console:Power off for gibba018 completed 2021-03-09T07:10:21.720 INFO:teuthology.orchestra.console:Power on gibba018 2021-03-09T07:10:30.117 INFO:teuthology.orchestra.console:Power on for gibba018 completed 2021-03-09T07:10:30.766 INFO:teuthology.provision.fog.gibba018:Waiting for deploy to finish 2021-03-09T07:15:20.690 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-09T07:15:21.085 INFO:teuthology.provision.fog.gibba018:Node is ready 2021-03-09T07:15:21.129 INFO:teuthology.orchestra.run.gibba018.stdout:gibba018.front.sepia.ceph.com 2021-03-09T07:15:21.173 INFO:teuthology.orchestra.run.gibba018.stdout:172.21.2.118 gibba018.front.sepia.ceph.com gibba018 2021-03-09T07:15:21.427 INFO:teuthology.provision.fog.gibba018:Deploy complete! 2021-03-09T07:15:21.532 INFO:teuthology.lock.ops:Checking gibba018.front.sepia.ceph.com 2021-03-09T07:15:21.538 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-09T07:15:21.558 INFO:teuthology.lock.ops:Updating [gibba018.front.sepia.ceph.com]: set os type and version on server 2021-03-09T07:15:21.558 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-09T07:15:21.834 INFO:teuthology.orchestra.run.gibba018.stdout:x86_64 2021-03-09T07:15:21.878 INFO:teuthology.orchestra.run.gibba018.stdout:NAME="Ubuntu" 2021-03-09T07:15:21.879 INFO:teuthology.orchestra.run.gibba018.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-03-09T07:15:21.879 INFO:teuthology.orchestra.run.gibba018.stdout:ID=ubuntu 2021-03-09T07:15:21.879 INFO:teuthology.orchestra.run.gibba018.stdout:ID_LIKE=debian 2021-03-09T07:15:21.880 INFO:teuthology.orchestra.run.gibba018.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-03-09T07:15:21.880 INFO:teuthology.orchestra.run.gibba018.stdout:VERSION_ID="18.04" 2021-03-09T07:15:21.880 INFO:teuthology.orchestra.run.gibba018.stdout:HOME_URL="https://www.ubuntu.com/" 2021-03-09T07:15:21.880 INFO:teuthology.orchestra.run.gibba018.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-03-09T07:15:21.881 INFO:teuthology.orchestra.run.gibba018.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-03-09T07:15:21.881 INFO:teuthology.orchestra.run.gibba018.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-03-09T07:15:21.881 INFO:teuthology.orchestra.run.gibba018.stdout:VERSION_CODENAME=bionic 2021-03-09T07:15:21.881 INFO:teuthology.orchestra.run.gibba018.stdout:UBUNTU_CODENAME=bionic 2021-03-09T07:15:21.883 INFO:teuthology.lock.ops:Updating gibba018.front.sepia.ceph.com on lock server 2021-03-09T07:15:22.270 INFO:teuthology.dispatcher.supervisor:Running job 5949068 2021-03-09T07:15:22.270 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/bin/teuthology -v --owner scheduled_bhubbard@teuthology --archive /home/teuthworker/archive/bhubbard-2021-03-09_06:19:15-rados:singleton-nomsgr:all:lazy_omap_stats_output.yaml-wip-badone-testing-distro-basic-gibba/5949068 --name bhubbard-2021-03-09_06:19:15-rados:singleton-nomsgr:all:lazy_omap_stats_output.yaml-wip-badone-testing-distro-basic-gibba --description rados:singleton-nomsgr:all:lazy_omap_stats_output.yaml -- /home/teuthworker/archive/bhubbard-2021-03-09_06:19:15-rados:singleton-nomsgr:all:lazy_omap_stats_output.yaml-wip-badone-testing-distro-basic-gibba/5949068/orig.config.yaml 2021-03-09T07:15:22.278 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/bhubbard-2021-03-09_06:19:15-rados:singleton-nomsgr:all:lazy_omap_stats_output.yaml-wip-badone-testing-distro-basic-gibba/5949068 2021-03-09T07:15:22.280 INFO:teuthology.dispatcher.supervisor:Job PID: 5465 2021-03-09T07:15:22.281 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-03-09T14:35:53.151 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/bhubbard-2021-03-09_06:19:15-rados:singleton-nomsgr:all:lazy_omap_stats_output.yaml-wip-badone-testing-distro-basic-gibba/jobs/ failed with status 500: Internal Server Error

Internal Server Error

2021-03-09T14:35:53.215 ERROR:teuthology.report:Could not report results to http://paddles.front.sepia.ceph.com/ Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/report.py", line 498, in try_push_job_info push_job_info(run_name, job_id, job_info) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/report.py", line 463, in push_job_info reporter.report_job(run_name, job_id, job_info) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/report.py", line 320, in report_job response.raise_for_status() File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/lib/python3.6/site-packages/requests/models.py", line 943, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http://paddles.front.sepia.ceph.com/runs/bhubbard-2021-03-09_06:19:15-rados:singleton-nomsgr:all:lazy_omap_stats_output.yaml-wip-badone-testing-distro-basic-gibba/jobs/ 2021-03-09T19:16:12.904 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-03-09T19:16:13.109 INFO:teuthology.kill:Killing Pids: {5465} 2021-03-09T19:16:15.050 INFO:teuthology.kill:Nuking machines: ['gibba018'] 2021-03-09T19:16:16.602 INFO:teuthology.kill:2021-03-09 19:16:16,602.602 INFO:teuthology.nuke:targets: 2021-03-09T19:16:16.606 INFO:teuthology.kill: gibba018.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDTZcgsEFpXVJfy1VjB7By7nb9zSIuEnt0EmWn4krpqzCArcdvvKBVu7aNOcsx2/5XqPJDBe5L4glBwrL9ykjtwZb/XzYnPEdWNJ8A+5eq/iVedWI6GJj+LC2IYIEumlKRBW1W0eWffI9/CtD1UUjrkvxhbzCN+CkOhx7Z8uPVMNnEY/yHwtRNXucOm+tno5Ep0M6gcnDa9QUkQVlOcuU5Pk0R+lzE7XEHf+EtgwLpCWG9+eDtbigkrejHZkaXLYxxrfxBHuEobIwAmnRSFgD055ZUXbn/u+y9d8v9B5wRiQmhg1DTt7IbF9fVnV6nVKYJBZd43PE2SootjA2oTqy+P 2021-03-09T19:16:16.606 INFO:teuthology.kill:2021-03-09 19:16:16,603.603 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-09T19:16:16.671 INFO:teuthology.kill:2021-03-09 19:16:16,670.670 INFO:teuthology.orchestra.console:Power off gibba018 2021-03-09T19:16:25.078 INFO:teuthology.kill:2021-03-09 19:16:25,078.078 INFO:teuthology.orchestra.console:Power off for gibba018 completed 2021-03-09T19:16:25.294 INFO:teuthology.task.internal:roles: ubuntu@gibba018.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2021-03-09T19:16:25.295 INFO:teuthology.misc:Compressing logs... 2021-03-09T19:16:25.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-09T19:16:43.842 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/misc.py", line 1374, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-03-09T19:16:43.883 INFO:teuthology.kill:No teuthology processes running 2021-03-09T19:16:45.402 INFO:teuthology.kill:Nuking machines: ['gibba018'] 2021-03-09T19:16:46.634 INFO:teuthology.kill:2021-03-09 19:16:46,634.634 INFO:teuthology.nuke:targets: 2021-03-09T19:16:46.635 INFO:teuthology.kill: gibba018.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDTZcgsEFpXVJfy1VjB7By7nb9zSIuEnt0EmWn4krpqzCArcdvvKBVu7aNOcsx2/5XqPJDBe5L4glBwrL9ykjtwZb/XzYnPEdWNJ8A+5eq/iVedWI6GJj+LC2IYIEumlKRBW1W0eWffI9/CtD1UUjrkvxhbzCN+CkOhx7Z8uPVMNnEY/yHwtRNXucOm+tno5Ep0M6gcnDa9QUkQVlOcuU5Pk0R+lzE7XEHf+EtgwLpCWG9+eDtbigkrejHZkaXLYxxrfxBHuEobIwAmnRSFgD055ZUXbn/u+y9d8v9B5wRiQmhg1DTt7IbF9fVnV6nVKYJBZd43PE2SootjA2oTqy+P 2021-03-09T19:16:46.657 INFO:teuthology.kill:2021-03-09 19:16:46,657.657 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-09T19:16:46.714 INFO:teuthology.kill:2021-03-09 19:16:46,714.714 INFO:teuthology.orchestra.console:Power off gibba018 2021-03-09T19:16:46.892 INFO:teuthology.kill:2021-03-09 19:16:46,892.892 INFO:teuthology.orchestra.console:Power off for gibba018 completed 2021-03-09T19:16:47.032 INFO:teuthology.kill:2021-03-09 19:16:47,031.031 INFO:teuthology.lock.ops:unlocked gibba018.front.sepia.ceph.com 2021-03-09T19:18:47.362 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-03-09T19:18:47.411 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-03-09T19:18:47.412 INFO:teuthology.nuke:Checking targets against current locks 2021-03-09T19:18:47.480 INFO:teuthology.nuke:Not nuking gibba018.front.sepia.ceph.com because description doesn't match