2021-04-16T00:41:03.582 INFO:root:teuthology version: 1.1.0-2713a3cd
2021-04-16T00:41:03.756 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging
2021-04-16T00:41:03.757 INFO:teuthology.lock.ops:Updating [ubuntu@smithi110.front.sepia.ceph.com]: reset os type and version on server
2021-04-16T00:41:03.758 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2021-04-16T00:41:03.784 INFO:teuthology.lock.ops:Node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging is complete
2021-04-16T00:41:03.785 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi161.front.sepia.ceph.com' reimaging
2021-04-16T00:41:03.785 INFO:teuthology.lock.ops:Updating [ubuntu@smithi161.front.sepia.ceph.com]: reset os type and version on server
2021-04-16T00:41:03.786 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server
2021-04-16T00:41:03.802 INFO:teuthology.lock.ops:Node 'ubuntu@smithi161.front.sepia.ceph.com' reimaging is complete
2021-04-16T00:41:04.030 INFO:teuthology.provision.fog.smithi110:Scheduling deploy of centos 8.3
2021-04-16T00:41:04.046 INFO:teuthology.provision.fog.smithi161:Scheduling deploy of centos 8.3
2021-04-16T00:41:04.556 INFO:teuthology.orchestra.console:Power off smithi161
2021-04-16T00:41:04.584 INFO:teuthology.orchestra.console:Power off smithi110
2021-04-16T00:41:12.954 INFO:teuthology.orchestra.console:Power off for smithi110 completed
2021-04-16T00:41:13.055 INFO:teuthology.orchestra.console:Power on smithi110
2021-04-16T00:41:17.056 INFO:teuthology.orchestra.console:Power off for smithi161 completed
2021-04-16T00:41:17.158 INFO:teuthology.orchestra.console:Power on smithi161
2021-04-16T00:41:21.475 INFO:teuthology.orchestra.console:Power on for smithi110 completed
2021-04-16T00:41:21.577 INFO:teuthology.provision.fog.smithi110:Waiting for deploy to finish
2021-04-16T00:41:25.530 INFO:teuthology.orchestra.console:Power on for smithi161 completed
2021-04-16T00:41:25.632 INFO:teuthology.provision.fog.smithi161:Waiting for deploy to finish
2021-04-16T00:45:44.359 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-04-16T00:45:44.659 INFO:teuthology.provision.fog.smithi110:Node is ready
2021-04-16T00:45:44.715 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110.front.sepia.ceph.com
2021-04-16T00:45:44.770 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110 smithi110.front.sepia.ceph.com smithi110
2021-04-16T00:45:45.084 INFO:teuthology.provision.fog.smithi110:Deploy complete!
2021-04-16T00:46:21.555 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-04-16T00:46:21.872 INFO:teuthology.provision.fog.smithi161:Node is ready
2021-04-16T00:46:21.887 INFO:teuthology.orchestra.run.smithi161.stdout:smithi161.front.sepia.ceph.com
2021-04-16T00:46:21.942 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161 smithi161.front.sepia.ceph.com smithi161
2021-04-16T00:46:22.277 INFO:teuthology.provision.fog.smithi161:Deploy complete!
2021-04-16T00:46:22.403 INFO:teuthology.lock.ops:Checking smithi110.front.sepia.ceph.com
2021-04-16T00:46:22.404 INFO:teuthology.lock.ops:New key found. Updating...
2021-04-16T00:46:22.420 INFO:teuthology.lock.ops:Checking smithi161.front.sepia.ceph.com
2021-04-16T00:46:22.420 INFO:teuthology.lock.ops:New key found. Updating...
2021-04-16T00:46:22.435 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: set os type and version on server
2021-04-16T00:46:22.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-04-16T00:46:22.724 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64
2021-04-16T00:46:22.780 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Linux"
2021-04-16T00:46:22.781 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="8"
2021-04-16T00:46:22.781 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos"
2021-04-16T00:46:22.781 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora"
2021-04-16T00:46:22.782 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="8"
2021-04-16T00:46:22.782 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el8"
2021-04-16T00:46:22.782 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Linux 8"
2021-04-16T00:46:22.782 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31"
2021-04-16T00:46:22.783 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-04-16T00:46:22.783 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/"
2021-04-16T00:46:22.783 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-04-16T00:46:22.783 INFO:teuthology.orchestra.run.smithi110.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-04-16T00:46:22.784 INFO:teuthology.orchestra.run.smithi110.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-04-16T00:46:22.785 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2021-04-16T00:46:22.801 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: set os type and version on server
2021-04-16T00:46:22.801 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-04-16T00:46:23.080 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64
2021-04-16T00:46:23.133 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Linux"
2021-04-16T00:46:23.134 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8"
2021-04-16T00:46:23.134 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos"
2021-04-16T00:46:23.134 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora"
2021-04-16T00:46:23.135 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8"
2021-04-16T00:46:23.135 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8"
2021-04-16T00:46:23.135 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Linux 8"
2021-04-16T00:46:23.136 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31"
2021-04-16T00:46:23.136 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-04-16T00:46:23.136 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/"
2021-04-16T00:46:23.137 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-04-16T00:46:23.137 INFO:teuthology.orchestra.run.smithi161.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-04-16T00:46:23.138 INFO:teuthology.orchestra.run.smithi161.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-04-16T00:46:23.138 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server
2021-04-16T00:46:23.219 INFO:teuthology.dispatcher.supervisor:Running job 6049893
2021-04-16T00:46:23.219 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/virtualenv/bin/teuthology -v --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2021-04-15_22:06:53-rados-wip-sage-testing-2021-04-15-1314-distro-basic-smithi/6049893 --name sage-2021-04-15_22:06:53-rados-wip-sage-testing-2021-04-15-1314-distro-basic-smithi --description rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} -- /home/teuthworker/archive/sage-2021-04-15_22:06:53-rados-wip-sage-testing-2021-04-15-1314-distro-basic-smithi/6049893/orig.config.yaml
2021-04-16T00:46:23.222 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sage-2021-04-15_22:06:53-rados-wip-sage-testing-2021-04-15-1314-distro-basic-smithi/6049893
2021-04-16T00:46:23.223 INFO:teuthology.dispatcher.supervisor:Job PID: 25791
2021-04-16T00:46:23.223 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-04-16T05:18:46.114 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/sage-2021-04-15_22:06:53-rados-wip-sage-testing-2021-04-15-1314-distro-basic-smithi/jobs/ failed with status 504: <html>
<head><title>504 Gateway Time-out</title></head>
<body bgcolor="white">
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>nginx/1.10.3 (Ubuntu)</center>
</body>
</html>

2021-04-16T05:46:50.918 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/sage-2021-04-15_22:06:53-rados-wip-sage-testing-2021-04-15-1314-distro-basic-smithi/jobs/ failed with status 504: <html>
<head><title>504 Gateway Time-out</title></head>
<body bgcolor="white">
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>nginx/1.10.3 (Ubuntu)</center>
</body>
</html>

2021-04-16T06:01:06.694 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/sage-2021-04-15_22:06:53-rados-wip-sage-testing-2021-04-15-1314-distro-basic-smithi/jobs/ failed with status 504: <html>
<head><title>504 Gateway Time-out</title></head>
<body bgcolor="white">
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>nginx/1.10.3 (Ubuntu)</center>
</body>
</html>

2021-04-16T06:02:07.738 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/sage-2021-04-15_22:06:53-rados-wip-sage-testing-2021-04-15-1314-distro-basic-smithi/jobs/ failed with status 504: <html>
<head><title>504 Gateway Time-out</title></head>
<body bgcolor="white">
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>nginx/1.10.3 (Ubuntu)</center>
</body>
</html>

2021-04-16T12:48:21.736 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-04-16T12:48:21.942 INFO:teuthology.kill:Killing Pids: {25791}
2021-04-16T12:48:23.185 INFO:teuthology.kill:Nuking machines: ['smithi110', 'smithi161']
2021-04-16T12:48:24.267 INFO:teuthology.kill:2021-04-16 12:48:24,266.266 INFO:teuthology.nuke:targets:
2021-04-16T12:48:24.268 INFO:teuthology.kill:  smithi110.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC3xp+G0V7evVuBX3CWnwBu8ddLrcaoFHA2wUab5m01A+nLiWRYD+WBRMqVz3NXl7ei1t+RyYO12TBmNruRiRzdFviXSsBhc2oj4h9sdSDRvYTUpRfDi8RdMANMHcjPqUEPSdZZ7BrLba2laYUNaKF+Bpi5BvtoXZTdX4cScW54JUYg7aS7NzRPQule8y/+THXaSbMULnEudpjX8EltqZjfdcM5pnuChbOKHYUisJblnmnAhT2HjdlOApiDcjELScE7uM63kuOBJ3WRQs/WhqQyF8nj/BsAutH1jWCec4CKI0pS+UloTDE/wYlkk+IK6AfCRCW7fpO+9VMTb4eHNFiKYW9Mjcwcuva5+yy0g5G+4gZdEPK7F5CbD3k5aRqIhbU+lhGlM5VtLtv7BsrS0N9KUz09MbE6JiEOPRJtkHObeSj5y0EhnQ8XWkz51/6y9ryg8fv+25j/ZK5pWfWH5FSTbjEiQ3VXMf5c3vvUC1wBLL2s2lJMEpZ3cBmiIG+an8=
2021-04-16T12:48:24.268 INFO:teuthology.kill:  smithi161.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0G5pKk+JW6oCALlTDxzSRlLYlioT1mXCAqSXO7IxCF7o+s56OjDyq8BlVVl+jxuxOoJxf8kfsmV9SRPuzKajH/2VIJ0YK7NJ/ats5qvIe/a/MeggscvtDxrdbXjZK9H7aLw/WwJqZ/qWAarYdoa2Dk5XYJREl1BQ1L4BA8UNT+Zsv4NuFjSX6JKY79vfRAZcOpgGIj6Y3zhxOd1h+dhGKYtk6hr9hU/jxKHtZQVqg3VB7o54F+9A5aRH451lLIiz/TEOM/uHW/eCz4KsPRt3uj3qHZslY37Rt+eHrcfnOXICTEC0Isk1p/B8qKC+pSH/ZCqZ3/8RQSTC2+Pqxdd0IX5nEYyQuAuGsIxyvPQGxN9vNevVs5GhZIcAF1thwnHJtyy6EoWlrZlNwBej0pgv2Bh23p+1QcqwguV00ri12Dek+wFYmJ7UVdWCEauofxNnzeXRT/gPVZ3MlK30kAWHIr2GbWb+CuZXAsBp3pBXydSbeBJx2HrjFHPFKal/D2j8=
2021-04-16T12:48:24.269 INFO:teuthology.kill:2021-04-16 12:48:24,267.267 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-16T12:48:24.269 INFO:teuthology.kill:2021-04-16 12:48:24,269.269 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-16T12:48:24.345 INFO:teuthology.kill:2021-04-16 12:48:24,345.345 INFO:teuthology.orchestra.console:Power off smithi110
2021-04-16T12:48:24.365 INFO:teuthology.kill:2021-04-16 12:48:24,364.364 INFO:teuthology.orchestra.console:Power off smithi161
2021-04-16T12:48:32.737 INFO:teuthology.kill:2021-04-16 12:48:32,736.736 INFO:teuthology.orchestra.console:Power off for smithi110 completed
2021-04-16T12:48:36.856 INFO:teuthology.kill:2021-04-16 12:48:36,856.856 INFO:teuthology.orchestra.console:Power off for smithi161 completed
2021-04-16T12:48:37.077 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2021-04-16T12:48:37.078 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2021-04-16T12:48:37.078 INFO:teuthology.misc:Compressing logs...
2021-04-16T12:48:37.079 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-04-16T12:49:11.772 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/remote.py", line 215, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-04-16T12:49:11.838 INFO:teuthology.kill:No teuthology processes running
2021-04-16T12:49:13.845 INFO:teuthology.kill:Nuking machines: ['smithi110', 'smithi161']
2021-04-16T12:49:14.877 INFO:teuthology.kill:2021-04-16 12:49:14,877.877 INFO:teuthology.nuke:targets:
2021-04-16T12:49:14.878 INFO:teuthology.kill:  smithi110.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDC3xp+G0V7evVuBX3CWnwBu8ddLrcaoFHA2wUab5m01A+nLiWRYD+WBRMqVz3NXl7ei1t+RyYO12TBmNruRiRzdFviXSsBhc2oj4h9sdSDRvYTUpRfDi8RdMANMHcjPqUEPSdZZ7BrLba2laYUNaKF+Bpi5BvtoXZTdX4cScW54JUYg7aS7NzRPQule8y/+THXaSbMULnEudpjX8EltqZjfdcM5pnuChbOKHYUisJblnmnAhT2HjdlOApiDcjELScE7uM63kuOBJ3WRQs/WhqQyF8nj/BsAutH1jWCec4CKI0pS+UloTDE/wYlkk+IK6AfCRCW7fpO+9VMTb4eHNFiKYW9Mjcwcuva5+yy0g5G+4gZdEPK7F5CbD3k5aRqIhbU+lhGlM5VtLtv7BsrS0N9KUz09MbE6JiEOPRJtkHObeSj5y0EhnQ8XWkz51/6y9ryg8fv+25j/ZK5pWfWH5FSTbjEiQ3VXMf5c3vvUC1wBLL2s2lJMEpZ3cBmiIG+an8=
2021-04-16T12:49:14.878 INFO:teuthology.kill:  smithi161.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0G5pKk+JW6oCALlTDxzSRlLYlioT1mXCAqSXO7IxCF7o+s56OjDyq8BlVVl+jxuxOoJxf8kfsmV9SRPuzKajH/2VIJ0YK7NJ/ats5qvIe/a/MeggscvtDxrdbXjZK9H7aLw/WwJqZ/qWAarYdoa2Dk5XYJREl1BQ1L4BA8UNT+Zsv4NuFjSX6JKY79vfRAZcOpgGIj6Y3zhxOd1h+dhGKYtk6hr9hU/jxKHtZQVqg3VB7o54F+9A5aRH451lLIiz/TEOM/uHW/eCz4KsPRt3uj3qHZslY37Rt+eHrcfnOXICTEC0Isk1p/B8qKC+pSH/ZCqZ3/8RQSTC2+Pqxdd0IX5nEYyQuAuGsIxyvPQGxN9vNevVs5GhZIcAF1thwnHJtyy6EoWlrZlNwBej0pgv2Bh23p+1QcqwguV00ri12Dek+wFYmJ7UVdWCEauofxNnzeXRT/gPVZ3MlK30kAWHIr2GbWb+CuZXAsBp3pBXydSbeBJx2HrjFHPFKal/D2j8=
2021-04-16T12:49:14.899 INFO:teuthology.kill:2021-04-16 12:49:14,899.899 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-16T12:49:14.912 INFO:teuthology.kill:2021-04-16 12:49:14,912.912 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-16T12:49:14.982 INFO:teuthology.kill:2021-04-16 12:49:14,982.982 INFO:teuthology.orchestra.console:Power off smithi110
2021-04-16T12:49:14.998 INFO:teuthology.kill:2021-04-16 12:49:14,998.998 INFO:teuthology.orchestra.console:Power off smithi161
2021-04-16T12:49:15.146 INFO:teuthology.kill:2021-04-16 12:49:15,146.146 INFO:teuthology.orchestra.console:Power off for smithi110 completed
2021-04-16T12:49:15.153 INFO:teuthology.kill:2021-04-16 12:49:15,153.153 INFO:teuthology.orchestra.console:Power off for smithi161 completed
2021-04-16T12:49:15.301 INFO:teuthology.kill:2021-04-16 12:49:15,300.300 INFO:teuthology.lock.ops:unlocked smithi110.front.sepia.ceph.com
2021-04-16T12:49:15.318 INFO:teuthology.kill:2021-04-16 12:49:15,317.317 INFO:teuthology.lock.ops:unlocked smithi161.front.sepia.ceph.com
2021-04-16T12:51:15.608 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-04-16T12:51:15.712 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-04-16T12:51:15.713 INFO:teuthology.nuke:Checking targets against current locks
2021-04-16T12:51:15.787 INFO:teuthology.nuke:Not nuking smithi161.front.sepia.ceph.com because description doesn't match
2021-04-16T12:51:15.787 INFO:teuthology.nuke:Not nuking smithi110.front.sepia.ceph.com because description doesn't match