2021-07-27T22:41:37.798 INFO:root:teuthology version: 1.1.0-73aa7e3b
2021-07-27T22:41:37.907 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi098.front.sepia.ceph.com' reimaging
2021-07-27T22:41:37.908 INFO:teuthology.lock.ops:Updating [ubuntu@smithi098.front.sepia.ceph.com]: reset os type and version on server
2021-07-27T22:41:37.909 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server
2021-07-27T22:41:37.933 INFO:teuthology.lock.ops:Node 'ubuntu@smithi098.front.sepia.ceph.com' reimaging is complete
2021-07-27T22:41:37.934 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi188.front.sepia.ceph.com' reimaging
2021-07-27T22:41:37.934 INFO:teuthology.lock.ops:Updating [ubuntu@smithi188.front.sepia.ceph.com]: reset os type and version on server
2021-07-27T22:41:37.935 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server
2021-07-27T22:41:37.953 INFO:teuthology.lock.ops:Node 'ubuntu@smithi188.front.sepia.ceph.com' reimaging is complete
2021-07-27T22:41:38.150 INFO:teuthology.provision.fog.smithi188:Scheduling deploy of centos 8.3
2021-07-27T22:41:38.162 INFO:teuthology.provision.fog.smithi098:Scheduling deploy of centos 8.3
2021-07-27T22:41:38.491 INFO:teuthology.orchestra.console:Power off smithi188
2021-07-27T22:41:38.507 INFO:teuthology.orchestra.console:Power off smithi098
2021-07-27T22:41:46.867 INFO:teuthology.orchestra.console:Power off for smithi188 completed
2021-07-27T22:41:46.969 INFO:teuthology.orchestra.console:Power on smithi188
2021-07-27T22:41:50.993 INFO:teuthology.orchestra.console:Power off for smithi098 completed
2021-07-27T22:41:51.095 INFO:teuthology.orchestra.console:Power on smithi098
2021-07-27T22:41:55.347 INFO:teuthology.orchestra.console:Power on for smithi188 completed
2021-07-27T22:41:55.351 INFO:teuthology.orchestra.console:Power on for smithi098 completed
2021-07-27T22:41:55.449 INFO:teuthology.provision.fog.smithi188:Waiting for deploy to finish
2021-07-27T22:41:55.453 INFO:teuthology.provision.fog.smithi098:Waiting for deploy to finish
2021-07-27T22:44:26.712 ERROR:teuthology.orchestra.connection:Error authenticating with smithi098.front.sepia.ceph.com: Authentication failed.
2021-07-27T22:46:27.212 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-27T22:46:27.560 INFO:teuthology.provision.fog.smithi188:Node is ready
2021-07-27T22:46:27.575 INFO:teuthology.orchestra.run.smithi188.stdout:smithi188.front.sepia.ceph.com
2021-07-27T22:46:27.629 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.15.188 smithi188.front.sepia.ceph.com smithi188
2021-07-27T22:46:27.982 INFO:teuthology.provision.fog.smithi188:Deploy complete!
2021-07-27T22:46:39.976 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-27T22:46:40.284 INFO:teuthology.provision.fog.smithi098:Node is ready
2021-07-27T22:46:40.298 INFO:teuthology.orchestra.run.smithi098.stdout:smithi098.front.sepia.ceph.com
2021-07-27T22:46:40.353 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98 smithi098.front.sepia.ceph.com smithi098
2021-07-27T22:46:40.665 INFO:teuthology.provision.fog.smithi098:Deploy complete!
2021-07-27T22:46:40.786 INFO:teuthology.lock.ops:Checking smithi098.front.sepia.ceph.com
2021-07-27T22:46:40.787 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-27T22:46:40.804 INFO:teuthology.lock.ops:Checking smithi188.front.sepia.ceph.com
2021-07-27T22:46:40.804 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-27T22:46:40.818 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: set os type and version on server
2021-07-27T22:46:40.819 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-27T22:46:41.095 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64
2021-07-27T22:46:41.148 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Linux"
2021-07-27T22:46:41.149 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8"
2021-07-27T22:46:41.149 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos"
2021-07-27T22:46:41.149 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora"
2021-07-27T22:46:41.149 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8"
2021-07-27T22:46:41.149 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8"
2021-07-27T22:46:41.150 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Linux 8"
2021-07-27T22:46:41.150 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31"
2021-07-27T22:46:41.150 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-07-27T22:46:41.150 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/"
2021-07-27T22:46:41.151 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-07-27T22:46:41.151 INFO:teuthology.orchestra.run.smithi098.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-07-27T22:46:41.151 INFO:teuthology.orchestra.run.smithi098.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-07-27T22:46:41.153 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server
2021-07-27T22:46:41.171 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: set os type and version on server
2021-07-27T22:46:41.171 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-27T22:46:41.462 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64
2021-07-27T22:46:41.516 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Linux"
2021-07-27T22:46:41.516 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="8"
2021-07-27T22:46:41.517 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos"
2021-07-27T22:46:41.517 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora"
2021-07-27T22:46:41.517 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="8"
2021-07-27T22:46:41.517 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el8"
2021-07-27T22:46:41.518 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Linux 8"
2021-07-27T22:46:41.518 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31"
2021-07-27T22:46:41.518 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-07-27T22:46:41.519 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/"
2021-07-27T22:46:41.519 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-07-27T22:46:41.519 INFO:teuthology.orchestra.run.smithi188.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-07-27T22:46:41.519 INFO:teuthology.orchestra.run.smithi188.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-07-27T22:46:41.520 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server
2021-07-27T22:46:41.594 INFO:teuthology.dispatcher.supervisor:Running job 6296473
2021-07-27T22:46:41.594 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-07-27_13:23:42-rados-wip_gbenhano_ncb_fix2-distro-basic-smithi/6296473 --name benhanokh-2021-07-27_13:23:42-rados-wip_gbenhano_ncb_fix2-distro-basic-smithi --description rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} -- /home/teuthworker/archive/benhanokh-2021-07-27_13:23:42-rados-wip_gbenhano_ncb_fix2-distro-basic-smithi/6296473/orig.config.yaml
2021-07-27T22:46:41.598 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-07-27_13:23:42-rados-wip_gbenhano_ncb_fix2-distro-basic-smithi/6296473
2021-07-27T22:46:41.599 INFO:teuthology.dispatcher.supervisor:Job PID: 10773
2021-07-27T22:46:41.599 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-07-28T10:46:58.228 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-07-28T10:46:58.472 INFO:teuthology.kill:Killing Pids: {10773}
2021-07-28T10:46:59.605 INFO:teuthology.kill:Nuking machines: ['smithi098', 'smithi188']
2021-07-28T10:47:00.595 INFO:teuthology.kill:2021-07-28 10:47:00,595.595 INFO:teuthology.nuke:targets:
2021-07-28T10:47:00.596 INFO:teuthology.kill:  smithi098.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCv647LVtAO5JLq7l6A9r0p4i0eNntJWT0a+ZusghIhA9OQ6oLyZfBhoeoE8qGgjfM2slwmCHEkbxRCIyipjHzF2xEt6pStPncMJzKLG+T4tE5H7J2/PgHQ4jhjicN+nnOdMaRvrbdUWCV9q9EZB0RUW+TeV35Q2uY6kdHK6ICepvhwgAy3Dfqh0bRlWoSVLQGR1qMl1Asjl+0LvyTzp46W/BS8mKkpGRAriZa+jzIUZpJ6O7W5lVAvQCqV8hHS0BnC6VpveuEGW+lEcNLnw20vAIHFEvmn2XBFyjPH0mtIdE+AH2/ZwquvuLxaUNGd9uCq8K++07FuJv+b7LxkfJmeJTuSNAGPL7J7C11GKnGxR/rwy0WW0NzrSO0Wz9CL4T7T0RonV9NCrk8G5rBsIdQ2iY0gRl94E3EFpitD7VdYQJ5B+xa/VfQo1sIMSz2lIEsW9uj7a8J7rfdouAsTpMofJ/dP62zqnogNoK+Kt85UfXire5tmmPdccUcK80xXA/c=
2021-07-28T10:47:00.596 INFO:teuthology.kill:  smithi188.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEVZDLordatUelsChfeckDTNw7lyzfCwjp/FhbrW3YAycYaCK8u3eAoqXpTMd2i2mrbcymWM6PeDqQdVThCV8ONZWuVmUJL6TnyIuU7l28iwmGHP6jzKeyGFGn7YhyPu5ifkItj6t42S8b8bKme5u29TzeoB5Wq+i2yGngGDT+LBYCfJl50pX07qupT3JT/K+ol+Z2Wh+BCmLBJ6vv+DD4z4XlXu6Ne7sZwPmHKVOH9zadPFXWxSil0KggtPayPUjDTr8pGjtnXwz+SeevJkVsTxHEqpkBB+T6Eje5t629qd423dR+7ULmbUo9H1SQKik8YuZ4SKrf6VmJYfZWEMEWNP813h2l0O5YavL+NOlzAHO7CH0eZsu0JiHZ53/EK9SX1nvo2Sw2Q+Bp8hQ2pDtx9RxsTQxMq1/Bw9Moj833DywWWdDGJ0+OwlKDUAFMj5AmfFWqykJfGwZCoFyUwmuN+mKdPv5fJRBKKyCtRL32kPaZrWxbWvhnfjTF83kNkGk=
2021-07-28T10:47:00.597 INFO:teuthology.kill:2021-07-28 10:47:00,595.595 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-28T10:47:00.598 INFO:teuthology.kill:2021-07-28 10:47:00,597.597 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-28T10:47:00.657 INFO:teuthology.kill:2021-07-28 10:47:00,656.656 INFO:teuthology.orchestra.console:Power off smithi188
2021-07-28T10:47:00.671 INFO:teuthology.kill:2021-07-28 10:47:00,671.671 INFO:teuthology.orchestra.console:Power off smithi098
2021-07-28T10:47:09.034 INFO:teuthology.kill:2021-07-28 10:47:09,033.033 INFO:teuthology.orchestra.console:Power off for smithi188 completed
2021-07-28T10:47:09.039 INFO:teuthology.kill:2021-07-28 10:47:09,038.038 INFO:teuthology.orchestra.console:Power off for smithi098 completed
2021-07-28T10:47:09.309 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2021-07-28T10:47:09.309 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2021-07-28T10:47:09.310 INFO:teuthology.misc:Compressing logs...
2021-07-28T10:47:09.311 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-28T10:48:09.314 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/remote.py", line 509, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-07-28T10:48:09.385 INFO:teuthology.kill:No teuthology processes running
2021-07-28T10:48:10.466 INFO:teuthology.kill:Nuking machines: ['smithi098', 'smithi188']
2021-07-28T10:48:11.520 INFO:teuthology.kill:2021-07-28 10:48:11,519.519 INFO:teuthology.nuke:targets:
2021-07-28T10:48:11.520 INFO:teuthology.kill:  smithi098.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCv647LVtAO5JLq7l6A9r0p4i0eNntJWT0a+ZusghIhA9OQ6oLyZfBhoeoE8qGgjfM2slwmCHEkbxRCIyipjHzF2xEt6pStPncMJzKLG+T4tE5H7J2/PgHQ4jhjicN+nnOdMaRvrbdUWCV9q9EZB0RUW+TeV35Q2uY6kdHK6ICepvhwgAy3Dfqh0bRlWoSVLQGR1qMl1Asjl+0LvyTzp46W/BS8mKkpGRAriZa+jzIUZpJ6O7W5lVAvQCqV8hHS0BnC6VpveuEGW+lEcNLnw20vAIHFEvmn2XBFyjPH0mtIdE+AH2/ZwquvuLxaUNGd9uCq8K++07FuJv+b7LxkfJmeJTuSNAGPL7J7C11GKnGxR/rwy0WW0NzrSO0Wz9CL4T7T0RonV9NCrk8G5rBsIdQ2iY0gRl94E3EFpitD7VdYQJ5B+xa/VfQo1sIMSz2lIEsW9uj7a8J7rfdouAsTpMofJ/dP62zqnogNoK+Kt85UfXire5tmmPdccUcK80xXA/c=
2021-07-28T10:48:11.521 INFO:teuthology.kill:  smithi188.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEVZDLordatUelsChfeckDTNw7lyzfCwjp/FhbrW3YAycYaCK8u3eAoqXpTMd2i2mrbcymWM6PeDqQdVThCV8ONZWuVmUJL6TnyIuU7l28iwmGHP6jzKeyGFGn7YhyPu5ifkItj6t42S8b8bKme5u29TzeoB5Wq+i2yGngGDT+LBYCfJl50pX07qupT3JT/K+ol+Z2Wh+BCmLBJ6vv+DD4z4XlXu6Ne7sZwPmHKVOH9zadPFXWxSil0KggtPayPUjDTr8pGjtnXwz+SeevJkVsTxHEqpkBB+T6Eje5t629qd423dR+7ULmbUo9H1SQKik8YuZ4SKrf6VmJYfZWEMEWNP813h2l0O5YavL+NOlzAHO7CH0eZsu0JiHZ53/EK9SX1nvo2Sw2Q+Bp8hQ2pDtx9RxsTQxMq1/Bw9Moj833DywWWdDGJ0+OwlKDUAFMj5AmfFWqykJfGwZCoFyUwmuN+mKdPv5fJRBKKyCtRL32kPaZrWxbWvhnfjTF83kNkGk=
2021-07-28T10:48:11.541 INFO:teuthology.kill:2021-07-28 10:48:11,540.540 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-28T10:48:11.544 INFO:teuthology.kill:2021-07-28 10:48:11,544.544 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-28T10:48:11.601 INFO:teuthology.kill:2021-07-28 10:48:11,601.601 INFO:teuthology.orchestra.console:Power off smithi098
2021-07-28T10:48:11.615 INFO:teuthology.kill:2021-07-28 10:48:11,615.615 INFO:teuthology.orchestra.console:Power off smithi188
2021-07-28T10:48:11.753 INFO:teuthology.kill:2021-07-28 10:48:11,753.753 INFO:teuthology.orchestra.console:Power off for smithi098 completed
2021-07-28T10:48:11.762 INFO:teuthology.kill:2021-07-28 10:48:11,762.762 INFO:teuthology.orchestra.console:Power off for smithi188 completed
2021-07-28T10:48:11.893 INFO:teuthology.kill:2021-07-28 10:48:11,892.892 INFO:teuthology.lock.ops:unlocked smithi098.front.sepia.ceph.com
2021-07-28T10:48:11.914 INFO:teuthology.kill:2021-07-28 10:48:11,913.913 INFO:teuthology.lock.ops:unlocked smithi188.front.sepia.ceph.com
2021-07-28T10:50:12.104 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-07-28T10:50:12.190 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-07-28T10:50:12.191 INFO:teuthology.nuke:Checking targets against current locks
2021-07-28T10:50:12.293 INFO:teuthology.nuke:Not nuking smithi098.front.sepia.ceph.com because description doesn't match
2021-07-28T10:50:12.294 INFO:teuthology.nuke:Not nuking smithi188.front.sepia.ceph.com because description doesn't match