2021-05-02T14:41:23.831 INFO:root:teuthology version: 1.1.0-19220a3b
2021-05-02T14:41:23.941 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi083.front.sepia.ceph.com' reimaging
2021-05-02T14:41:23.942 INFO:teuthology.lock.ops:Updating [ubuntu@smithi083.front.sepia.ceph.com]: reset os type and version on server
2021-05-02T14:41:23.943 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server
2021-05-02T14:41:23.960 INFO:teuthology.lock.ops:Node 'ubuntu@smithi083.front.sepia.ceph.com' reimaging is complete
2021-05-02T14:41:23.960 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi187.front.sepia.ceph.com' reimaging
2021-05-02T14:41:23.961 INFO:teuthology.lock.ops:Updating [ubuntu@smithi187.front.sepia.ceph.com]: reset os type and version on server
2021-05-02T14:41:23.961 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server
2021-05-02T14:41:23.977 INFO:teuthology.lock.ops:Node 'ubuntu@smithi187.front.sepia.ceph.com' reimaging is complete
2021-05-02T14:41:24.215 INFO:teuthology.provision.fog.smithi187:Scheduling deploy of rhel 8.3
2021-05-02T14:41:24.225 INFO:teuthology.provision.fog.smithi083:Scheduling deploy of rhel 8.3
2021-05-02T14:41:24.625 INFO:teuthology.orchestra.console:Power off smithi187
2021-05-02T14:41:24.724 INFO:teuthology.orchestra.console:Power off smithi083
2021-05-02T14:41:33.065 INFO:teuthology.orchestra.console:Power off for smithi187 completed
2021-05-02T14:41:33.166 INFO:teuthology.orchestra.console:Power on smithi187
2021-05-02T14:41:37.224 INFO:teuthology.orchestra.console:Power off for smithi083 completed
2021-05-02T14:41:37.325 INFO:teuthology.orchestra.console:Power on smithi083
2021-05-02T14:41:41.546 INFO:teuthology.orchestra.console:Power on for smithi187 completed
2021-05-02T14:41:41.576 INFO:teuthology.orchestra.console:Power on for smithi083 completed
2021-05-02T14:41:41.647 INFO:teuthology.provision.fog.smithi187:Waiting for deploy to finish
2021-05-02T14:41:41.676 INFO:teuthology.provision.fog.smithi083:Waiting for deploy to finish
2021-05-02T14:46:13.644 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-05-02T14:46:13.669 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-05-02T14:46:13.908 INFO:teuthology.provision.fog.smithi187:Node is ready
2021-05-02T14:46:13.923 INFO:teuthology.orchestra.run.smithi187.stdout:smithi187.front.sepia.ceph.com
2021-05-02T14:46:13.971 INFO:teuthology.provision.fog.smithi083:Node is ready
2021-05-02T14:46:13.977 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.15.187 smithi187.front.sepia.ceph.com smithi187
2021-05-02T14:46:13.985 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083.front.sepia.ceph.com
2021-05-02T14:46:14.040 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83 smithi083.front.sepia.ceph.com smithi083
2021-05-02T14:46:14.303 INFO:teuthology.provision.fog.smithi187:Deploy complete!
2021-05-02T14:46:14.360 INFO:teuthology.provision.fog.smithi083:Deploy complete!
2021-05-02T14:46:14.503 INFO:teuthology.lock.ops:Checking smithi083.front.sepia.ceph.com
2021-05-02T14:46:14.507 INFO:teuthology.lock.ops:New key found. Updating...
2021-05-02T14:46:14.525 INFO:teuthology.lock.ops:Checking smithi187.front.sepia.ceph.com
2021-05-02T14:46:14.525 INFO:teuthology.lock.ops:New key found. Updating...
2021-05-02T14:46:14.540 INFO:teuthology.lock.ops:Updating [smithi083.front.sepia.ceph.com]: set os type and version on server
2021-05-02T14:46:14.541 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-05-02T14:46:14.819 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64
2021-05-02T14:46:14.873 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="Red Hat Enterprise Linux"
2021-05-02T14:46:14.873 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="8.3 (Ootpa)"
2021-05-02T14:46:14.874 INFO:teuthology.orchestra.run.smithi083.stdout:ID="rhel"
2021-05-02T14:46:14.874 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="fedora"
2021-05-02T14:46:14.874 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="8.3"
2021-05-02T14:46:14.874 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el8"
2021-05-02T14:46:14.875 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-05-02T14:46:14.875 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31"
2021-05-02T14:46:14.875 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-05-02T14:46:14.875 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://www.redhat.com/"
2021-05-02T14:46:14.875 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-05-02T14:46:14.876 INFO:teuthology.orchestra.run.smithi083.stdout:
2021-05-02T14:46:14.876 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-05-02T14:46:14.876 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-05-02T14:46:14.876 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-05-02T14:46:14.877 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-05-02T14:46:14.878 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server
2021-05-02T14:46:14.896 INFO:teuthology.lock.ops:Updating [smithi187.front.sepia.ceph.com]: set os type and version on server
2021-05-02T14:46:14.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-05-02T14:46:15.174 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64
2021-05-02T14:46:15.228 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="Red Hat Enterprise Linux"
2021-05-02T14:46:15.229 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="8.3 (Ootpa)"
2021-05-02T14:46:15.229 INFO:teuthology.orchestra.run.smithi187.stdout:ID="rhel"
2021-05-02T14:46:15.229 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="fedora"
2021-05-02T14:46:15.229 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="8.3"
2021-05-02T14:46:15.230 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el8"
2021-05-02T14:46:15.230 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-05-02T14:46:15.230 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31"
2021-05-02T14:46:15.230 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-05-02T14:46:15.231 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.redhat.com/"
2021-05-02T14:46:15.231 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-05-02T14:46:15.231 INFO:teuthology.orchestra.run.smithi187.stdout:
2021-05-02T14:46:15.232 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-05-02T14:46:15.232 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-05-02T14:46:15.232 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-05-02T14:46:15.232 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-05-02T14:46:15.233 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server
2021-05-02T14:46:15.313 INFO:teuthology.dispatcher.supervisor:Running job 6087916
2021-05-02T14:46:15.314 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2021-05-01_09:07:09-fs-wip-pdonnell-testing-20210501.040415-distro-basic-smithi/6087916 --name pdonnell-2021-05-01_09:07:09-fs-wip-pdonnell-testing-20210501.040415-distro-basic-smithi --description fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/fs/snaps}} -- /home/teuthworker/archive/pdonnell-2021-05-01_09:07:09-fs-wip-pdonnell-testing-20210501.040415-distro-basic-smithi/6087916/orig.config.yaml
2021-05-02T14:46:15.317 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2021-05-01_09:07:09-fs-wip-pdonnell-testing-20210501.040415-distro-basic-smithi/6087916
2021-05-02T14:46:15.318 INFO:teuthology.dispatcher.supervisor:Job PID: 16768
2021-05-02T14:46:15.318 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-05-03T02:47:07.416 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-05-03T02:47:07.659 INFO:teuthology.kill:Killing Pids: {16768}
2021-05-03T02:47:09.396 INFO:teuthology.kill:Nuking machines: ['smithi083', 'smithi187']
2021-05-03T02:47:10.426 INFO:teuthology.kill:2021-05-03 02:47:10,425.425 INFO:teuthology.nuke:targets:
2021-05-03T02:47:10.426 INFO:teuthology.kill:  smithi083.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFYSLhL0pdmBZjc6VoJyD8NZVl8dx0o0soUvZV4n84XAsq8pQbYlHVjPw6t8ismuWNF28gD24bac+c1nagiYJj6uFjGmDzEzWeAZGL6xCnfOceiOq7x/cIfT6g7oMavoK4bEGb529RO2kscqF/b98e1HIvXs7zG6rFaYoT1SO9L75PoDK6zGxeHMVV+mDXP9qxdu3Ao9tUnmyxResFDk0783DhL6MhWFmtB3muu5KZKg6/UKem5rMut+tRttCijvBJJU0MOKwCCe4Grt7zRnHjwJofMC0L/d+sQcdbLFDZ4rFtLq04d8x2ldJ3IebvRxI2vYezBAfWXYSqmRxp79YgKVRMLseMK7yBalrAtJf5PuzJHOOaPeBEIQix8IDqXZcAGuB6vZG2qbFc5eHq/Xchha+in1D9wcE/l+gMNNATS58HR7KA0hrfDNqMjbbDOPHk1mYuGGUB9ZRWnkRFKgYFIOfC1aA8O5mMznoamD97eLgCmSN9UHnHtT0SPb6kDEk=
2021-05-03T02:47:10.427 INFO:teuthology.kill:  smithi187.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC56uEGChYFsNHq/2QNAR4zheWdNn2yj384q/LB5mObedre7tBZhzGrebfca1rrWU3fDWQU9xPy5KUxE9N+8azPgUiMl2kmw2KDaAmmMdESLOA5B1T3Q4DygVGv+ZHij4Ix28te+DQCnhbtVRgqpEbMouKqs88iQ2wZRo8iyzl5GnRsWP3nTRXt8JNqahJNnLARI5i56hFFpLECeSCYDI1rob1wjZqs3qvI2/xPh9rJotzn8AhnD4gTHNCHfw5Bc1GMOkT146ynufAzPgvVKRAkwA+r+HaC0xAGyZfNVdRsoZb/DEYFg3nhurZbkcW6do370+VNi60g+E6IYjzJIFFiEcFSg6fgjHZLtJHgQXKQijTOpUHxSu3QONxO/Z6Tl6bD+UUgvqR9Q3dKbASwsTWjxZLFNdX3yBy4UI8//mnwQuSJEIC04HdLOHLMbtAecuqJr9f6tGBOH7UGCv7iHCcVNps0BMGaHHsdM0iseaaBvakoVDc6A1Vvpc4DxT9C9mM=
2021-05-03T02:47:10.427 INFO:teuthology.kill:2021-05-03 02:47:10,426.426 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-03T02:47:10.428 INFO:teuthology.kill:2021-05-03 02:47:10,428.428 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-03T02:47:10.490 INFO:teuthology.kill:2021-05-03 02:47:10,489.489 INFO:teuthology.orchestra.console:Power off smithi083
2021-05-03T02:47:10.526 INFO:teuthology.kill:2021-05-03 02:47:10,526.526 INFO:teuthology.orchestra.console:Power off smithi187
2021-05-03T02:47:18.884 INFO:teuthology.kill:2021-05-03 02:47:18,884.884 INFO:teuthology.orchestra.console:Power off for smithi083 completed
2021-05-03T02:47:23.001 INFO:teuthology.kill:2021-05-03 02:47:23,000.000 INFO:teuthology.orchestra.console:Power off for smithi187 completed
2021-05-03T02:47:23.280 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2021-05-03T02:47:23.281 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2021-05-03T02:47:23.281 INFO:teuthology.misc:Compressing logs...
2021-05-03T02:47:23.283 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-05-03T02:48:23.344 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/remote.py", line 509, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-05-03T02:48:23.419 INFO:teuthology.kill:No teuthology processes running
2021-05-03T02:48:25.256 INFO:teuthology.kill:Nuking machines: ['smithi083', 'smithi187']
2021-05-03T02:48:26.787 INFO:teuthology.kill:2021-05-03 02:48:26,786.786 INFO:teuthology.nuke:targets:
2021-05-03T02:48:26.789 INFO:teuthology.kill:  smithi083.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFYSLhL0pdmBZjc6VoJyD8NZVl8dx0o0soUvZV4n84XAsq8pQbYlHVjPw6t8ismuWNF28gD24bac+c1nagiYJj6uFjGmDzEzWeAZGL6xCnfOceiOq7x/cIfT6g7oMavoK4bEGb529RO2kscqF/b98e1HIvXs7zG6rFaYoT1SO9L75PoDK6zGxeHMVV+mDXP9qxdu3Ao9tUnmyxResFDk0783DhL6MhWFmtB3muu5KZKg6/UKem5rMut+tRttCijvBJJU0MOKwCCe4Grt7zRnHjwJofMC0L/d+sQcdbLFDZ4rFtLq04d8x2ldJ3IebvRxI2vYezBAfWXYSqmRxp79YgKVRMLseMK7yBalrAtJf5PuzJHOOaPeBEIQix8IDqXZcAGuB6vZG2qbFc5eHq/Xchha+in1D9wcE/l+gMNNATS58HR7KA0hrfDNqMjbbDOPHk1mYuGGUB9ZRWnkRFKgYFIOfC1aA8O5mMznoamD97eLgCmSN9UHnHtT0SPb6kDEk=
2021-05-03T02:48:26.790 INFO:teuthology.kill:  smithi187.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC56uEGChYFsNHq/2QNAR4zheWdNn2yj384q/LB5mObedre7tBZhzGrebfca1rrWU3fDWQU9xPy5KUxE9N+8azPgUiMl2kmw2KDaAmmMdESLOA5B1T3Q4DygVGv+ZHij4Ix28te+DQCnhbtVRgqpEbMouKqs88iQ2wZRo8iyzl5GnRsWP3nTRXt8JNqahJNnLARI5i56hFFpLECeSCYDI1rob1wjZqs3qvI2/xPh9rJotzn8AhnD4gTHNCHfw5Bc1GMOkT146ynufAzPgvVKRAkwA+r+HaC0xAGyZfNVdRsoZb/DEYFg3nhurZbkcW6do370+VNi60g+E6IYjzJIFFiEcFSg6fgjHZLtJHgQXKQijTOpUHxSu3QONxO/Z6Tl6bD+UUgvqR9Q3dKbASwsTWjxZLFNdX3yBy4UI8//mnwQuSJEIC04HdLOHLMbtAecuqJr9f6tGBOH7UGCv7iHCcVNps0BMGaHHsdM0iseaaBvakoVDc6A1Vvpc4DxT9C9mM=
2021-05-03T02:48:26.813 INFO:teuthology.kill:2021-05-03 02:48:26,813.813 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-03T02:48:26.816 INFO:teuthology.kill:2021-05-03 02:48:26,815.815 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-03T02:48:26.876 INFO:teuthology.kill:2021-05-03 02:48:26,876.876 INFO:teuthology.orchestra.console:Power off smithi187
2021-05-03T02:48:26.890 INFO:teuthology.kill:2021-05-03 02:48:26,889.889 INFO:teuthology.orchestra.console:Power off smithi083
2021-05-03T02:48:27.021 INFO:teuthology.kill:2021-05-03 02:48:27,021.021 INFO:teuthology.orchestra.console:Power off for smithi187 completed
2021-05-03T02:48:27.028 INFO:teuthology.kill:2021-05-03 02:48:27,028.028 INFO:teuthology.orchestra.console:Power off for smithi083 completed
2021-05-03T02:48:27.162 INFO:teuthology.kill:2021-05-03 02:48:27,162.162 INFO:teuthology.lock.ops:unlocked smithi187.front.sepia.ceph.com
2021-05-03T02:48:27.168 INFO:teuthology.kill:2021-05-03 02:48:27,167.167 INFO:teuthology.lock.ops:unlocked smithi083.front.sepia.ceph.com
2021-05-03T02:50:27.476 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-05-03T02:50:27.587 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-05-03T02:50:27.588 INFO:teuthology.nuke:Checking targets against current locks
2021-05-03T02:50:27.711 INFO:teuthology.nuke:Not nuking smithi187.front.sepia.ceph.com because description doesn't match
2021-05-03T02:50:27.712 INFO:teuthology.nuke:Not nuking smithi083.front.sepia.ceph.com because description doesn't match