2021-03-15T23:44:25.264 INFO:root:teuthology version: 1.1.0-b9656917
2021-03-15T23:44:25.374 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging
2021-03-15T23:44:25.375 INFO:teuthology.lock.ops:Updating [ubuntu@smithi170.front.sepia.ceph.com]: reset os type and version on server
2021-03-15T23:44:25.375 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server
2021-03-15T23:44:25.391 INFO:teuthology.lock.ops:Node 'ubuntu@smithi170.front.sepia.ceph.com' reimaging is complete
2021-03-15T23:44:25.391 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi173.front.sepia.ceph.com' reimaging
2021-03-15T23:44:25.391 INFO:teuthology.lock.ops:Updating [ubuntu@smithi173.front.sepia.ceph.com]: reset os type and version on server
2021-03-15T23:44:25.392 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server
2021-03-15T23:44:25.407 INFO:teuthology.lock.ops:Node 'ubuntu@smithi173.front.sepia.ceph.com' reimaging is complete
2021-03-15T23:44:25.590 INFO:teuthology.provision.fog.smithi170:Scheduling deploy of centos 8.3
2021-03-15T23:44:25.599 INFO:teuthology.provision.fog.smithi173:Scheduling deploy of centos 8.3
2021-03-15T23:44:25.950 INFO:teuthology.orchestra.console:Power off smithi170
2021-03-15T23:44:25.964 INFO:teuthology.orchestra.console:Power off smithi173
2021-03-15T23:44:26.094 INFO:teuthology.orchestra.console:Power off for smithi170 completed
2021-03-15T23:44:26.107 INFO:teuthology.orchestra.console:Power off for smithi173 completed
2021-03-15T23:44:26.196 INFO:teuthology.orchestra.console:Power on smithi170
2021-03-15T23:44:26.208 INFO:teuthology.orchestra.console:Power on smithi173
2021-03-15T23:44:30.454 INFO:teuthology.orchestra.console:Power on for smithi170 completed
2021-03-15T23:44:30.556 INFO:teuthology.provision.fog.smithi170:Waiting for deploy to finish
2021-03-15T23:44:34.580 INFO:teuthology.orchestra.console:Power on for smithi173 completed
2021-03-15T23:44:34.681 INFO:teuthology.provision.fog.smithi173:Waiting for deploy to finish
2021-03-15T23:49:06.651 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-03-15T23:49:06.984 INFO:teuthology.provision.fog.smithi173:Node is ready
2021-03-15T23:49:07.001 INFO:teuthology.orchestra.run.smithi173.stdout:smithi173.front.sepia.ceph.com
2021-03-15T23:49:07.056 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.15.173 smithi173.front.sepia.ceph.com smithi173
2021-03-15T23:49:07.402 INFO:teuthology.provision.fog.smithi173:Deploy complete!
2021-03-15T23:49:08.559 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-03-15T23:49:08.876 INFO:teuthology.provision.fog.smithi170:Node is ready
2021-03-15T23:49:08.891 INFO:teuthology.orchestra.run.smithi170.stdout:smithi170.front.sepia.ceph.com
2021-03-15T23:49:08.945 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170 smithi170.front.sepia.ceph.com smithi170
2021-03-15T23:49:09.278 INFO:teuthology.provision.fog.smithi170:Deploy complete!
2021-03-15T23:49:09.428 INFO:teuthology.lock.ops:Checking smithi170.front.sepia.ceph.com
2021-03-15T23:49:09.435 INFO:teuthology.lock.ops:New key found. Updating...
2021-03-15T23:49:09.452 INFO:teuthology.lock.ops:Checking smithi173.front.sepia.ceph.com
2021-03-15T23:49:09.452 INFO:teuthology.lock.ops:New key found. Updating...
2021-03-15T23:49:09.479 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: set os type and version on server
2021-03-15T23:49:09.480 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-03-15T23:49:09.762 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64
2021-03-15T23:49:09.816 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Linux"
2021-03-15T23:49:09.817 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8"
2021-03-15T23:49:09.817 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos"
2021-03-15T23:49:09.817 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora"
2021-03-15T23:49:09.817 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8"
2021-03-15T23:49:09.818 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8"
2021-03-15T23:49:09.818 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Linux 8"
2021-03-15T23:49:09.818 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31"
2021-03-15T23:49:09.818 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-03-15T23:49:09.818 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/"
2021-03-15T23:49:09.819 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-03-15T23:49:09.819 INFO:teuthology.orchestra.run.smithi170.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-03-15T23:49:09.819 INFO:teuthology.orchestra.run.smithi170.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-03-15T23:49:09.821 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server
2021-03-15T23:49:09.835 INFO:teuthology.lock.ops:Updating [smithi173.front.sepia.ceph.com]: set os type and version on server
2021-03-15T23:49:09.836 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-03-15T23:49:10.127 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64
2021-03-15T23:49:10.182 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Linux"
2021-03-15T23:49:10.182 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="8"
2021-03-15T23:49:10.182 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos"
2021-03-15T23:49:10.183 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora"
2021-03-15T23:49:10.183 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="8"
2021-03-15T23:49:10.183 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el8"
2021-03-15T23:49:10.183 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Linux 8"
2021-03-15T23:49:10.184 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31"
2021-03-15T23:49:10.184 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-03-15T23:49:10.184 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/"
2021-03-15T23:49:10.184 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-03-15T23:49:10.184 INFO:teuthology.orchestra.run.smithi173.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-03-15T23:49:10.185 INFO:teuthology.orchestra.run.smithi173.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-03-15T23:49:10.185 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server
2021-03-15T23:49:10.272 INFO:teuthology.dispatcher.supervisor:Running job 5968090
2021-03-15T23:49:10.272 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2021-03-15_22:16:56-fs-wip-pdonnell-testing-20210315.182203-distro-basic-smithi/5968090 --name pdonnell-2021-03-15_22:16:56-fs-wip-pdonnell-testing-20210315.182203-distro-basic-smithi --description fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} -- /home/teuthworker/archive/pdonnell-2021-03-15_22:16:56-fs-wip-pdonnell-testing-20210315.182203-distro-basic-smithi/5968090/orig.config.yaml
2021-03-15T23:49:10.276 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2021-03-15_22:16:56-fs-wip-pdonnell-testing-20210315.182203-distro-basic-smithi/5968090
2021-03-15T23:49:10.277 INFO:teuthology.dispatcher.supervisor:Job PID: 26516
2021-03-15T23:49:10.277 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-03-16T06:37:39.041 ERROR:teuthology.report:Could not report results to http://paddles.front.sepia.ceph.com/
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/report.py", line 498, in try_push_job_info
    push_job_info(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/report.py", line 463, in push_job_info
    reporter.report_job(run_name, job_id, job_info)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/report.py", line 320, in report_job
    response.raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/lib/python3.6/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http://paddles.front.sepia.ceph.com/runs/pdonnell-2021-03-15_22:16:56-fs-wip-pdonnell-testing-20210315.182203-distro-basic-smithi/jobs/5968090/
2021-03-16T11:50:01.061 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-03-16T11:50:01.300 INFO:teuthology.kill:Killing Pids: {26516}
2021-03-16T11:50:02.344 INFO:teuthology.kill:Nuking machines: ['smithi170', 'smithi173']
2021-03-16T11:50:03.270 INFO:teuthology.kill:2021-03-16 11:50:03,270.270 INFO:teuthology.nuke:targets:
2021-03-16T11:50:03.271 INFO:teuthology.kill:  smithi170.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuecMzt4GK0bpUWh6eSY8hz65hZnA6dKQJTnZzfBb/zzMzY83SgR7OIV9Thm7L6Eybx9LssYdYXJNeTae4pfr6a1sjKV3uIRvHFx13q8zZwWmt/Rg8tgKStuMZWPaT3+Uo4pzS3FgSqsaAuFNLBwb1CP3xigh0RC6WfbVw4CvtWFpADzypWMzeAXDM3mKctCVumLn5ELf38yLyl5+MEf4T8kCaoCzVtC9iDvKYC7bdk7iY4dFnDrCJQSAjEy2jUHenLueJAN3IJIIUzEssUoxJUiaT9oe3mKYYr/ekbTOXghUPPSy79t1vpRHim7TPTeOONIlnDzIFSAUVCcwyL6HPx0SJ3AvXyWD9rhqhZnvu3sleZvsR4QYXko+3VnGKaGCy3qgY9NxoSn7nVne3u1/546Xvq0uNGlSMoDjZQw5zg6BuYJPttcPbkJBnS0miJWrA+2p6FabPAIiWNP/GYdOwO4wA6+16rUM9zpGZiDP/LwNQoxL5PqVrtcOPrsPqwzc=
2021-03-16T11:50:03.271 INFO:teuthology.kill:  smithi173.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1+Afm0Hr59swYeAUZd90JoNiIYjKfsVAwhbv49+rpAEILrQ3o0Mc113HmSnJnIJbeSA++3Jj06ApTCgoO5mMnhJm+HOJjGejmdUGwpAma1Gud4lvu4UrBgiZMLGLj+ai5f3NrjkKFnWcybILoXtMch6W6i6rjpc7ISydP4vdfuSX0v5vozsJdVcitn+kikp3CvPlHzQ2lmnBbeoiSrCw2b28qLrKjG3OST5RG++kKbxpaXDHEZcURB1pOdEwxzZSTD3C0UF0OSzDVqOmp4Rb0uHqFcgkSAQ9Vu1KXTyw+SwC+bIjaijNhZr1yHYyEPIbB2aGER6qH5PmmEaT+vTCz53aJ4McCZaBAuCPECKqMBhMyKJn1rgukyYttg/tG5lv2Nx1AyTFcnQSyZjKwj1Rg7QP0qOOZ/H256XDWAkWnZlVnSCVlB2KtRNaXzpPXNFYk712GKNrf2YgeHBwEqChZUIF11TxqcqApHwqYuOHa2eC9hEiuVONKhF6jDjdjrHM=
2021-03-16T11:50:03.272 INFO:teuthology.kill:2021-03-16 11:50:03,270.270 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-03-16T11:50:03.273 INFO:teuthology.kill:2021-03-16 11:50:03,273.273 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-03-16T11:50:03.333 INFO:teuthology.kill:2021-03-16 11:50:03,333.333 INFO:teuthology.orchestra.console:Power off smithi170
2021-03-16T11:50:03.345 INFO:teuthology.kill:2021-03-16 11:50:03,344.344 INFO:teuthology.orchestra.console:Power off smithi173
2021-03-16T11:50:11.709 INFO:teuthology.kill:2021-03-16 11:50:11,708.708 INFO:teuthology.orchestra.console:Power off for smithi173 completed
2021-03-16T11:50:15.811 INFO:teuthology.kill:2021-03-16 11:50:15,811.811 INFO:teuthology.orchestra.console:Power off for smithi170 completed
2021-03-16T11:50:16.244 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2021-03-16T11:50:16.245 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2021-03-16T11:50:16.245 INFO:teuthology.misc:Compressing logs...
2021-03-16T11:50:16.246 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-03-16T11:50:16.742 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/misc.py", line 1374, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/remote.py", line 215, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-03-16T11:50:16.811 INFO:teuthology.kill:No teuthology processes running
2021-03-16T11:50:17.875 INFO:teuthology.kill:Nuking machines: ['smithi170', 'smithi173']
2021-03-16T11:50:18.822 INFO:teuthology.kill:2021-03-16 11:50:18,822.822 INFO:teuthology.nuke:targets:
2021-03-16T11:50:18.823 INFO:teuthology.kill:  smithi170.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuecMzt4GK0bpUWh6eSY8hz65hZnA6dKQJTnZzfBb/zzMzY83SgR7OIV9Thm7L6Eybx9LssYdYXJNeTae4pfr6a1sjKV3uIRvHFx13q8zZwWmt/Rg8tgKStuMZWPaT3+Uo4pzS3FgSqsaAuFNLBwb1CP3xigh0RC6WfbVw4CvtWFpADzypWMzeAXDM3mKctCVumLn5ELf38yLyl5+MEf4T8kCaoCzVtC9iDvKYC7bdk7iY4dFnDrCJQSAjEy2jUHenLueJAN3IJIIUzEssUoxJUiaT9oe3mKYYr/ekbTOXghUPPSy79t1vpRHim7TPTeOONIlnDzIFSAUVCcwyL6HPx0SJ3AvXyWD9rhqhZnvu3sleZvsR4QYXko+3VnGKaGCy3qgY9NxoSn7nVne3u1/546Xvq0uNGlSMoDjZQw5zg6BuYJPttcPbkJBnS0miJWrA+2p6FabPAIiWNP/GYdOwO4wA6+16rUM9zpGZiDP/LwNQoxL5PqVrtcOPrsPqwzc=
2021-03-16T11:50:18.824 INFO:teuthology.kill:  smithi173.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1+Afm0Hr59swYeAUZd90JoNiIYjKfsVAwhbv49+rpAEILrQ3o0Mc113HmSnJnIJbeSA++3Jj06ApTCgoO5mMnhJm+HOJjGejmdUGwpAma1Gud4lvu4UrBgiZMLGLj+ai5f3NrjkKFnWcybILoXtMch6W6i6rjpc7ISydP4vdfuSX0v5vozsJdVcitn+kikp3CvPlHzQ2lmnBbeoiSrCw2b28qLrKjG3OST5RG++kKbxpaXDHEZcURB1pOdEwxzZSTD3C0UF0OSzDVqOmp4Rb0uHqFcgkSAQ9Vu1KXTyw+SwC+bIjaijNhZr1yHYyEPIbB2aGER6qH5PmmEaT+vTCz53aJ4McCZaBAuCPECKqMBhMyKJn1rgukyYttg/tG5lv2Nx1AyTFcnQSyZjKwj1Rg7QP0qOOZ/H256XDWAkWnZlVnSCVlB2KtRNaXzpPXNFYk712GKNrf2YgeHBwEqChZUIF11TxqcqApHwqYuOHa2eC9hEiuVONKhF6jDjdjrHM=
2021-03-16T11:50:18.844 INFO:teuthology.kill:2021-03-16 11:50:18,843.843 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-03-16T11:50:18.846 INFO:teuthology.kill:2021-03-16 11:50:18,846.846 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-03-16T11:50:18.902 INFO:teuthology.kill:2021-03-16 11:50:18,902.902 INFO:teuthology.orchestra.console:Power off smithi173
2021-03-16T11:50:18.916 INFO:teuthology.kill:2021-03-16 11:50:18,916.916 INFO:teuthology.orchestra.console:Power off smithi170
2021-03-16T11:50:19.049 INFO:teuthology.kill:2021-03-16 11:50:19,048.048 INFO:teuthology.orchestra.console:Power off for smithi173 completed
2021-03-16T11:50:19.060 INFO:teuthology.kill:2021-03-16 11:50:19,060.060 INFO:teuthology.orchestra.console:Power off for smithi170 completed
2021-03-16T11:50:19.189 INFO:teuthology.kill:2021-03-16 11:50:19,188.188 INFO:teuthology.lock.ops:unlocked smithi173.front.sepia.ceph.com
2021-03-16T11:50:19.196 INFO:teuthology.kill:2021-03-16 11:50:19,195.195 ERROR:teuthology.lock.ops:failed to unlock smithi170.front.sepia.ceph.com. reason: 500
2021-03-16T11:52:19.487 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-03-16T11:52:19.591 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-03-16T11:52:19.592 INFO:teuthology.nuke:Checking targets against current locks
2021-03-16T11:52:19.662 INFO:teuthology.nuke:Not nuking smithi173.front.sepia.ceph.com because description doesn't match
2021-03-16T11:52:19.680 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-03-16T11:52:19.732 INFO:teuthology.orchestra.console:Power off smithi170
2021-03-16T11:52:19.873 INFO:teuthology.orchestra.console:Power off for smithi170 completed
2021-03-16T11:52:20.016 INFO:teuthology.lock.ops:unlocked smithi170.front.sepia.ceph.com