2021-05-14T02:53:25.415 INFO:root:teuthology version: 1.1.0-19220a3b
2021-05-14T02:53:25.524 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi067.front.sepia.ceph.com' reimaging
2021-05-14T02:53:25.526 INFO:teuthology.lock.ops:Updating [ubuntu@smithi067.front.sepia.ceph.com]: reset os type and version on server
2021-05-14T02:53:25.526 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server
2021-05-14T02:53:25.541 INFO:teuthology.lock.ops:Node 'ubuntu@smithi067.front.sepia.ceph.com' reimaging is complete
2021-05-14T02:53:25.542 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi171.front.sepia.ceph.com' reimaging
2021-05-14T02:53:25.542 INFO:teuthology.lock.ops:Updating [ubuntu@smithi171.front.sepia.ceph.com]: reset os type and version on server
2021-05-14T02:53:25.542 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server
2021-05-14T02:53:25.557 INFO:teuthology.lock.ops:Node 'ubuntu@smithi171.front.sepia.ceph.com' reimaging is complete
2021-05-14T02:53:25.731 INFO:teuthology.provision.fog.smithi067:Scheduling deploy of centos 8.3
2021-05-14T02:53:25.755 INFO:teuthology.provision.fog.smithi171:Scheduling deploy of centos 8.3
2021-05-14T02:53:26.073 INFO:teuthology.orchestra.console:Power off smithi067
2021-05-14T02:53:26.110 INFO:teuthology.orchestra.console:Power off smithi171
2021-05-14T02:53:34.470 INFO:teuthology.orchestra.console:Power off for smithi171 completed
2021-05-14T02:53:34.571 INFO:teuthology.orchestra.console:Power on smithi171
2021-05-14T02:53:38.554 INFO:teuthology.orchestra.console:Power off for smithi067 completed
2021-05-14T02:53:38.656 INFO:teuthology.orchestra.console:Power on smithi067
2021-05-14T02:53:42.908 INFO:teuthology.orchestra.console:Power on for smithi067 completed
2021-05-14T02:53:42.930 INFO:teuthology.orchestra.console:Power on for smithi171 completed
2021-05-14T02:53:43.010 INFO:teuthology.provision.fog.smithi067:Waiting for deploy to finish
2021-05-14T02:53:43.032 INFO:teuthology.provision.fog.smithi171:Waiting for deploy to finish
2021-05-14T02:58:29.994 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-05-14T02:58:30.104 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-05-14T02:58:30.302 INFO:teuthology.provision.fog.smithi067:Node is ready
2021-05-14T02:58:30.318 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067.front.sepia.ceph.com
2021-05-14T02:58:30.374 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67 smithi067.front.sepia.ceph.com smithi067
2021-05-14T02:58:30.424 INFO:teuthology.provision.fog.smithi171:Node is ready
2021-05-14T02:58:30.439 INFO:teuthology.orchestra.run.smithi171.stdout:smithi171.front.sepia.ceph.com
2021-05-14T02:58:30.494 INFO:teuthology.orchestra.run.smithi171.stdout:172.21.15.171 smithi171.front.sepia.ceph.com smithi171
2021-05-14T02:58:30.705 INFO:teuthology.provision.fog.smithi067:Deploy complete!
2021-05-14T02:58:30.813 INFO:teuthology.provision.fog.smithi171:Deploy complete!
2021-05-14T02:58:30.941 INFO:teuthology.lock.ops:Checking smithi067.front.sepia.ceph.com
2021-05-14T02:58:30.942 INFO:teuthology.lock.ops:New key found. Updating...
2021-05-14T02:58:30.959 INFO:teuthology.lock.ops:Checking smithi171.front.sepia.ceph.com
2021-05-14T02:58:30.960 INFO:teuthology.lock.ops:New key found. Updating...
2021-05-14T02:58:30.977 INFO:teuthology.lock.ops:Updating [smithi067.front.sepia.ceph.com]: set os type and version on server
2021-05-14T02:58:30.978 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-05-14T02:58:31.264 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64
2021-05-14T02:58:31.318 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Linux"
2021-05-14T02:58:31.319 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="8"
2021-05-14T02:58:31.319 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos"
2021-05-14T02:58:31.319 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora"
2021-05-14T02:58:31.319 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="8"
2021-05-14T02:58:31.320 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el8"
2021-05-14T02:58:31.320 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Linux 8"
2021-05-14T02:58:31.320 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31"
2021-05-14T02:58:31.320 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-05-14T02:58:31.321 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/"
2021-05-14T02:58:31.321 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-05-14T02:58:31.321 INFO:teuthology.orchestra.run.smithi067.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-05-14T02:58:31.321 INFO:teuthology.orchestra.run.smithi067.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-05-14T02:58:31.323 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server
2021-05-14T02:58:31.339 INFO:teuthology.lock.ops:Updating [smithi171.front.sepia.ceph.com]: set os type and version on server
2021-05-14T02:58:31.339 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-05-14T02:58:31.623 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64
2021-05-14T02:58:31.677 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Linux"
2021-05-14T02:58:31.677 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="8"
2021-05-14T02:58:31.677 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos"
2021-05-14T02:58:31.678 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora"
2021-05-14T02:58:31.678 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="8"
2021-05-14T02:58:31.678 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el8"
2021-05-14T02:58:31.678 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Linux 8"
2021-05-14T02:58:31.679 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31"
2021-05-14T02:58:31.679 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-05-14T02:58:31.679 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/"
2021-05-14T02:58:31.679 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-05-14T02:58:31.679 INFO:teuthology.orchestra.run.smithi171.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-05-14T02:58:31.680 INFO:teuthology.orchestra.run.smithi171.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-05-14T02:58:31.680 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server
2021-05-14T02:58:31.755 INFO:teuthology.dispatcher.supervisor:Running job 6113775
2021-05-14T02:58:31.756 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-05-13_15:35:24-rados-no_column_b-distro-basic-smithi/6113775 --name benhanokh-2021-05-13_15:35:24-rados-no_column_b-distro-basic-smithi --description rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} -- /home/teuthworker/archive/benhanokh-2021-05-13_15:35:24-rados-no_column_b-distro-basic-smithi/6113775/orig.config.yaml
2021-05-14T02:58:31.759 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-05-13_15:35:24-rados-no_column_b-distro-basic-smithi/6113775
2021-05-14T02:58:31.760 INFO:teuthology.dispatcher.supervisor:Job PID: 15823
2021-05-14T02:58:31.760 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-05-14T14:59:18.191 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-05-14T14:59:18.360 INFO:teuthology.kill:Killing Pids: {15823}
2021-05-14T14:59:19.297 INFO:teuthology.kill:Nuking machines: ['smithi067', 'smithi171']
2021-05-14T14:59:20.118 INFO:teuthology.kill:2021-05-14 14:59:20,118.118 INFO:teuthology.nuke:targets:
2021-05-14T14:59:20.119 INFO:teuthology.kill:  smithi067.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSKRFOf4iMXQ8YWZjZwhrIRenkwiqdW5jUrgFAYpxpynkW8KTJgF8kcjPLw8F0GOWPfXn/kq28KD+a/ucolAnEw8vTsur3SjWNzge0J2bqiJsNYSbspjJasFNWw+aPF+2s0M4Z+HEHkY0alG3rBhViakUVBb3rOH0LPcpWqfn8Rv4rFUoyqCy2NXgfSTWv2ncUNa3U5wG0mO0kpWyJL7aWRBnJ/ohaE77DXg5WdWW2NhHCl2ti0w+bQ7dDQkFUmrHaOgMHRzoptSdGbqXSpK7gmaEUM/1hrhnbXKjQSB1lZFubrE/2rSH5GPaFcjjUPMHJFEKfthZsxDRlsKCVuFomK2tdE5dLnsPqManPZJhxsKx/6UoB5Aegyznr/e6hWBfBMzxSHQ7fYYdu9G8J3h1qnxqo+NDAK3wPJu3dJbaAcwiPosaOykumhHP9wZ2myn8o6w5s2heJ3gNG7yW1TXoHRxKau2ssEC2wOK1xjb2ZRlYRumxHvOaJBIzGvYkP0nU=
2021-05-14T14:59:20.119 INFO:teuthology.kill:  smithi171.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDardWvKlOZ3+QazYKNrq1CxpGGM+qG2tMfYyvksu+uwF49iEav0m08Ubknh68FdT0qCFfcI+LpJ1/5UKIxOtjLZ7g0RIrp2oeMntuBKmYub0mp9fGXLU5bf2JZgw4PzaiL1Hmcdq7qV3BlfKLHeCYLnlkXgglti6pjQqTo71ctp2i4KE4Wa8GBiDmnmPvkBsiRbdGer9VaNa7YjjC5nFwM1fpYbWv73J/Q7HK1gk8WX2npCOmK6bR3Do2fcCSE2FxqiDOK2hDLzR29on3LQN7qqlKHYVMm+pEfhdXXJ8o29GvvE9fkULo8D+dVJseLFHetiMZN5xmlk3Yr147q42Z59KTMn6SvbZMk9BhY0tebiEjfcg3XPBz/cEE0q77NAWxG0OhMOeZ8CojLUBZ30VPiIOTb1DXPMZ8YKAuG+EmCkdFOCXXlF7QCPrhDcqXpxw/lgoyhkocKtBnvDdlA1ziEP2FNvibDWFhQG7F9y8kz7+YucBOGbX4+gvROLTbR5TE=
2021-05-14T14:59:20.119 INFO:teuthology.kill:2021-05-14 14:59:20,118.118 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-14T14:59:20.120 INFO:teuthology.kill:2021-05-14 14:59:20,120.120 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-14T14:59:20.174 INFO:teuthology.kill:2021-05-14 14:59:20,173.173 INFO:teuthology.orchestra.console:Power off smithi067
2021-05-14T14:59:20.185 INFO:teuthology.kill:2021-05-14 14:59:20,185.185 INFO:teuthology.orchestra.console:Power off smithi171
2021-05-14T14:59:28.554 INFO:teuthology.kill:2021-05-14 14:59:28,553.553 INFO:teuthology.orchestra.console:Power off for smithi171 completed
2021-05-14T14:59:32.671 INFO:teuthology.kill:2021-05-14 14:59:32,670.670 INFO:teuthology.orchestra.console:Power off for smithi067 completed
2021-05-14T14:59:32.874 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2021-05-14T14:59:32.875 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2021-05-14T14:59:32.875 INFO:teuthology.misc:Compressing logs...
2021-05-14T14:59:32.876 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-05-14T15:00:07.499 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/remote.py", line 509, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-05-14T15:00:07.553 INFO:teuthology.kill:No teuthology processes running
2021-05-14T15:00:08.519 INFO:teuthology.kill:Nuking machines: ['smithi067', 'smithi171']
2021-05-14T15:00:09.411 INFO:teuthology.kill:2021-05-14 15:00:09,411.411 INFO:teuthology.nuke:targets:
2021-05-14T15:00:09.412 INFO:teuthology.kill:  smithi067.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSKRFOf4iMXQ8YWZjZwhrIRenkwiqdW5jUrgFAYpxpynkW8KTJgF8kcjPLw8F0GOWPfXn/kq28KD+a/ucolAnEw8vTsur3SjWNzge0J2bqiJsNYSbspjJasFNWw+aPF+2s0M4Z+HEHkY0alG3rBhViakUVBb3rOH0LPcpWqfn8Rv4rFUoyqCy2NXgfSTWv2ncUNa3U5wG0mO0kpWyJL7aWRBnJ/ohaE77DXg5WdWW2NhHCl2ti0w+bQ7dDQkFUmrHaOgMHRzoptSdGbqXSpK7gmaEUM/1hrhnbXKjQSB1lZFubrE/2rSH5GPaFcjjUPMHJFEKfthZsxDRlsKCVuFomK2tdE5dLnsPqManPZJhxsKx/6UoB5Aegyznr/e6hWBfBMzxSHQ7fYYdu9G8J3h1qnxqo+NDAK3wPJu3dJbaAcwiPosaOykumhHP9wZ2myn8o6w5s2heJ3gNG7yW1TXoHRxKau2ssEC2wOK1xjb2ZRlYRumxHvOaJBIzGvYkP0nU=
2021-05-14T15:00:09.412 INFO:teuthology.kill:  smithi171.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDardWvKlOZ3+QazYKNrq1CxpGGM+qG2tMfYyvksu+uwF49iEav0m08Ubknh68FdT0qCFfcI+LpJ1/5UKIxOtjLZ7g0RIrp2oeMntuBKmYub0mp9fGXLU5bf2JZgw4PzaiL1Hmcdq7qV3BlfKLHeCYLnlkXgglti6pjQqTo71ctp2i4KE4Wa8GBiDmnmPvkBsiRbdGer9VaNa7YjjC5nFwM1fpYbWv73J/Q7HK1gk8WX2npCOmK6bR3Do2fcCSE2FxqiDOK2hDLzR29on3LQN7qqlKHYVMm+pEfhdXXJ8o29GvvE9fkULo8D+dVJseLFHetiMZN5xmlk3Yr147q42Z59KTMn6SvbZMk9BhY0tebiEjfcg3XPBz/cEE0q77NAWxG0OhMOeZ8CojLUBZ30VPiIOTb1DXPMZ8YKAuG+EmCkdFOCXXlF7QCPrhDcqXpxw/lgoyhkocKtBnvDdlA1ziEP2FNvibDWFhQG7F9y8kz7+YucBOGbX4+gvROLTbR5TE=
2021-05-14T15:00:09.430 INFO:teuthology.kill:2021-05-14 15:00:09,430.430 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-14T15:00:09.433 INFO:teuthology.kill:2021-05-14 15:00:09,433.433 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-14T15:00:09.486 INFO:teuthology.kill:2021-05-14 15:00:09,485.485 INFO:teuthology.orchestra.console:Power off smithi171
2021-05-14T15:00:09.498 INFO:teuthology.kill:2021-05-14 15:00:09,498.498 INFO:teuthology.orchestra.console:Power off smithi067
2021-05-14T15:00:09.629 INFO:teuthology.kill:2021-05-14 15:00:09,629.629 INFO:teuthology.orchestra.console:Power off for smithi171 completed
2021-05-14T15:00:09.640 INFO:teuthology.kill:2021-05-14 15:00:09,640.640 INFO:teuthology.orchestra.console:Power off for smithi067 completed
2021-05-14T15:00:09.768 INFO:teuthology.kill:2021-05-14 15:00:09,767.767 INFO:teuthology.lock.ops:unlocked smithi171.front.sepia.ceph.com
2021-05-14T15:00:09.775 INFO:teuthology.kill:2021-05-14 15:00:09,774.774 INFO:teuthology.lock.ops:unlocked smithi067.front.sepia.ceph.com
2021-05-14T15:02:10.022 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15