2021-07-26T12:22:45.071 INFO:root:teuthology version: 1.1.0-b6f3efab
2021-07-26T12:22:45.163 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi154.front.sepia.ceph.com' reimaging
2021-07-26T12:22:45.164 INFO:teuthology.lock.ops:Updating [ubuntu@smithi154.front.sepia.ceph.com]: reset os type and version on server
2021-07-26T12:22:45.165 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server
2021-07-26T12:22:45.181 INFO:teuthology.lock.ops:Node 'ubuntu@smithi154.front.sepia.ceph.com' reimaging is complete
2021-07-26T12:22:45.182 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi174.front.sepia.ceph.com' reimaging
2021-07-26T12:22:45.182 INFO:teuthology.lock.ops:Updating [ubuntu@smithi174.front.sepia.ceph.com]: reset os type and version on server
2021-07-26T12:22:45.182 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server
2021-07-26T12:22:45.202 INFO:teuthology.lock.ops:Node 'ubuntu@smithi174.front.sepia.ceph.com' reimaging is complete
2021-07-26T12:22:45.389 INFO:teuthology.provision.fog.smithi154:Scheduling deploy of centos 8.2
2021-07-26T12:22:45.461 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of centos 8.2
2021-07-26T12:22:45.716 INFO:teuthology.orchestra.console:Power off smithi154
2021-07-26T12:22:45.831 INFO:teuthology.orchestra.console:Power off smithi174
2021-07-26T12:22:54.095 INFO:teuthology.orchestra.console:Power off for smithi154 completed
2021-07-26T12:22:54.194 INFO:teuthology.orchestra.console:Power off for smithi174 completed
2021-07-26T12:22:54.197 INFO:teuthology.orchestra.console:Power on smithi154
2021-07-26T12:22:54.295 INFO:teuthology.orchestra.console:Power on smithi174
2021-07-26T12:23:02.556 INFO:teuthology.orchestra.console:Power on for smithi154 completed
2021-07-26T12:23:02.649 INFO:teuthology.orchestra.console:Power on for smithi174 completed
2021-07-26T12:23:02.657 INFO:teuthology.provision.fog.smithi154:Waiting for deploy to finish
2021-07-26T12:23:02.751 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish
2021-07-26T12:27:41.090 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-26T12:27:41.410 INFO:teuthology.provision.fog.smithi174:Node is ready
2021-07-26T12:27:41.424 INFO:teuthology.orchestra.run.smithi174.stdout:smithi174.front.sepia.ceph.com
2021-07-26T12:27:41.479 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.174 smithi174.front.sepia.ceph.com smithi174
2021-07-26T12:27:41.809 INFO:teuthology.provision.fog.smithi174:Deploy complete!
2021-07-26T12:27:56.222 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-26T12:27:56.534 INFO:teuthology.provision.fog.smithi154:Node is ready
2021-07-26T12:27:56.548 INFO:teuthology.orchestra.run.smithi154.stdout:smithi154.front.sepia.ceph.com
2021-07-26T12:27:56.603 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154 smithi154.front.sepia.ceph.com smithi154
2021-07-26T12:27:56.916 INFO:teuthology.provision.fog.smithi154:Deploy complete!
2021-07-26T12:27:57.030 INFO:teuthology.lock.ops:Checking smithi154.front.sepia.ceph.com
2021-07-26T12:27:57.031 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-26T12:27:57.047 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com
2021-07-26T12:27:57.047 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-26T12:27:57.063 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: set os type and version on server
2021-07-26T12:27:57.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-26T12:27:57.325 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64
2021-07-26T12:27:57.379 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Linux"
2021-07-26T12:27:57.379 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="8 (Core)"
2021-07-26T12:27:57.379 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos"
2021-07-26T12:27:57.379 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora"
2021-07-26T12:27:57.380 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="8"
2021-07-26T12:27:57.380 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el8"
2021-07-26T12:27:57.380 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Linux 8 (Core)"
2021-07-26T12:27:57.380 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31"
2021-07-26T12:27:57.380 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-07-26T12:27:57.381 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://www.centos.org/"
2021-07-26T12:27:57.381 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-07-26T12:27:57.381 INFO:teuthology.orchestra.run.smithi154.stdout:
2021-07-26T12:27:57.381 INFO:teuthology.orchestra.run.smithi154.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-07-26T12:27:57.382 INFO:teuthology.orchestra.run.smithi154.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-07-26T12:27:57.382 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="centos"
2021-07-26T12:27:57.382 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8"
2021-07-26T12:27:57.382 INFO:teuthology.orchestra.run.smithi154.stdout:
2021-07-26T12:27:57.383 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server
2021-07-26T12:27:57.396 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server
2021-07-26T12:27:57.397 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-26T12:27:57.667 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64
2021-07-26T12:27:57.721 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Linux"
2021-07-26T12:27:57.722 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="8 (Core)"
2021-07-26T12:27:57.722 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos"
2021-07-26T12:27:57.722 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora"
2021-07-26T12:27:57.722 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="8"
2021-07-26T12:27:57.723 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el8"
2021-07-26T12:27:57.723 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Linux 8 (Core)"
2021-07-26T12:27:57.723 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31"
2021-07-26T12:27:57.723 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-07-26T12:27:57.724 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://www.centos.org/"
2021-07-26T12:27:57.724 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-07-26T12:27:57.724 INFO:teuthology.orchestra.run.smithi174.stdout:
2021-07-26T12:27:57.724 INFO:teuthology.orchestra.run.smithi174.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-07-26T12:27:57.724 INFO:teuthology.orchestra.run.smithi174.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-07-26T12:27:57.725 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="centos"
2021-07-26T12:27:57.725 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8"
2021-07-26T12:27:57.725 INFO:teuthology.orchestra.run.smithi174.stdout:
2021-07-26T12:27:57.726 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server
2021-07-26T12:27:57.792 INFO:teuthology.dispatcher.supervisor:Running job 6293454
2021-07-26T12:27:57.792 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-07-26_12:21:56-rados:cephadm:thrash-WIP_GBH_obj_scrubber2-distro-basic-smithi/6293454 --name benhanokh-2021-07-26_12:21:56-rados:cephadm:thrash-WIP_GBH_obj_scrubber2-distro-basic-smithi --description rados:cephadm:thrash/{0-distro/centos_8.2_kubic_stable 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} -- /home/teuthworker/archive/benhanokh-2021-07-26_12:21:56-rados:cephadm:thrash-WIP_GBH_obj_scrubber2-distro-basic-smithi/6293454/orig.config.yaml
2021-07-26T12:27:57.795 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-07-26_12:21:56-rados:cephadm:thrash-WIP_GBH_obj_scrubber2-distro-basic-smithi/6293454
2021-07-26T12:27:57.796 INFO:teuthology.dispatcher.supervisor:Job PID: 22038
2021-07-26T12:27:57.796 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-07-27T00:28:48.743 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-07-27T00:28:48.877 INFO:teuthology.kill:Killing Pids: {22038}
2021-07-27T00:28:50.216 INFO:teuthology.kill:Nuking machines: ['smithi154', 'smithi174']
2021-07-27T00:28:51.359 INFO:teuthology.kill:2021-07-27 00:28:51,355.355 INFO:teuthology.nuke:targets:
2021-07-27T00:28:51.360 INFO:teuthology.kill:  smithi154.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPSyQg6oYryRFTsuM9XCCqB+YUd/taOZADGexwhklYtBVOfrKzCklTRad59pfJOORjzaRjdO1C6isrMvo/hr8lpQXWRMbSUKigz7bm2sXBtj5n0SmjShed0yaEw+B0xhsKZmwCS3ANo5+TFfvTd7gzoHv6Ogs0+OIVzF6zS5Bx0nYl7/djQxZsLh1FUGQPexk4kk+8GdVWDDI7JgwuCinIFH20DYUooIyLJYvEDnjRNwVWVWA6fz2H8SBi75WS8GSds/iR5TGmjSslrvA55OY86vU3AkxELVcshgVaLYMifQ4R6SiQOJnHZbKe58RPYv3CYN861NcUUchvOluw4gXDwaNXE0erpBMvze6rmpaebldqXnIyzOJrzULAgSy+zjQ6szqQkZ2tInu4ufDsvU5GbkSgUgQPFzroWIiJZH5FcohGA58wHqXIAiHWCiFrjQrVgSpf885glsyvSBXPp4ux+HM5ee7VmgnPuNjdVk/dgDUwqwb/ZUZtVjO0bNMYtps=
2021-07-27T00:28:51.360 INFO:teuthology.kill:  smithi174.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5fp+5zmt0avMiIJrjSRt4PGl5YEEiMygkyzWrUBHgsNT+TiyUSdk91GOYrASqAalDRm1Jq+1/0Vz59D4uJOnbyWC2qpabW1E/s4FeDay0ERZaZogY6/oWIlau4mQmJaQz3Jfm5+jSWVJw0kKy6/uOQRJP3PX6z86TKxmZfmA0J6yRqn3182XL7MMfLNNmJpg7IE+W2krKBHvnwUHVByG7g/O4FVooRRFNK8v09cNuah5NBT3TzJMZ6N/ccdrDvUYXmqmODTAT4haxabhA+f1fslgTD8CRmt4DJP2Z8+jszo84/8uWzOTrMXfFN73hcg26cDo80pUb1pkIIxzTlfZ7L1DBOGLgU+AseYB9b7pZD3z+1JIwUs7geYLetWzBp+qLBnhX57hm0uvESCswuwA164uvKoSG/xSgyCz5L9VwuqxO/UH1ujhgOrtWvA6s8LoslmCHBS7z8CzdKoch7omGmE6BvNtIrDkR9k8kGwqyfZ3qtHmKQdxMJTZQLum1w2U=
2021-07-27T00:28:51.360 INFO:teuthology.kill:2021-07-27 00:28:51,355.355 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-27T00:28:51.361 INFO:teuthology.kill:2021-07-27 00:28:51,358.358 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-27T00:28:51.455 INFO:teuthology.kill:2021-07-27 00:28:51,454.454 INFO:teuthology.orchestra.console:Power off smithi174
2021-07-27T00:28:51.495 INFO:teuthology.kill:2021-07-27 00:28:51,494.494 INFO:teuthology.orchestra.console:Power off smithi154
2021-07-27T00:29:03.985 INFO:teuthology.kill:2021-07-27 00:29:03,984.984 INFO:teuthology.orchestra.console:Power off for smithi174 completed
2021-07-27T00:29:03.999 INFO:teuthology.kill:2021-07-27 00:29:03,998.998 INFO:teuthology.orchestra.console:Power off for smithi154 completed
2021-07-27T00:29:04.289 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a']
2021-07-27T00:29:04.290 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a']
2021-07-27T00:29:04.291 INFO:teuthology.misc:Compressing logs...
2021-07-27T00:29:04.291 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-27T00:29:38.891 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/remote.py", line 509, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-07-27T00:29:38.942 INFO:teuthology.kill:No teuthology processes running
2021-07-27T00:29:40.061 INFO:teuthology.kill:Nuking machines: ['smithi154', 'smithi174']
2021-07-27T00:29:41.174 INFO:teuthology.kill:2021-07-27 00:29:41,174.174 INFO:teuthology.nuke:targets:
2021-07-27T00:29:41.175 INFO:teuthology.kill:  smithi154.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPSyQg6oYryRFTsuM9XCCqB+YUd/taOZADGexwhklYtBVOfrKzCklTRad59pfJOORjzaRjdO1C6isrMvo/hr8lpQXWRMbSUKigz7bm2sXBtj5n0SmjShed0yaEw+B0xhsKZmwCS3ANo5+TFfvTd7gzoHv6Ogs0+OIVzF6zS5Bx0nYl7/djQxZsLh1FUGQPexk4kk+8GdVWDDI7JgwuCinIFH20DYUooIyLJYvEDnjRNwVWVWA6fz2H8SBi75WS8GSds/iR5TGmjSslrvA55OY86vU3AkxELVcshgVaLYMifQ4R6SiQOJnHZbKe58RPYv3CYN861NcUUchvOluw4gXDwaNXE0erpBMvze6rmpaebldqXnIyzOJrzULAgSy+zjQ6szqQkZ2tInu4ufDsvU5GbkSgUgQPFzroWIiJZH5FcohGA58wHqXIAiHWCiFrjQrVgSpf885glsyvSBXPp4ux+HM5ee7VmgnPuNjdVk/dgDUwqwb/ZUZtVjO0bNMYtps=
2021-07-27T00:29:41.176 INFO:teuthology.kill:  smithi174.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5fp+5zmt0avMiIJrjSRt4PGl5YEEiMygkyzWrUBHgsNT+TiyUSdk91GOYrASqAalDRm1Jq+1/0Vz59D4uJOnbyWC2qpabW1E/s4FeDay0ERZaZogY6/oWIlau4mQmJaQz3Jfm5+jSWVJw0kKy6/uOQRJP3PX6z86TKxmZfmA0J6yRqn3182XL7MMfLNNmJpg7IE+W2krKBHvnwUHVByG7g/O4FVooRRFNK8v09cNuah5NBT3TzJMZ6N/ccdrDvUYXmqmODTAT4haxabhA+f1fslgTD8CRmt4DJP2Z8+jszo84/8uWzOTrMXfFN73hcg26cDo80pUb1pkIIxzTlfZ7L1DBOGLgU+AseYB9b7pZD3z+1JIwUs7geYLetWzBp+qLBnhX57hm0uvESCswuwA164uvKoSG/xSgyCz5L9VwuqxO/UH1ujhgOrtWvA6s8LoslmCHBS7z8CzdKoch7omGmE6BvNtIrDkR9k8kGwqyfZ3qtHmKQdxMJTZQLum1w2U=
2021-07-27T00:29:41.199 INFO:teuthology.kill:2021-07-27 00:29:41,197.197 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-27T00:29:41.206 INFO:teuthology.kill:2021-07-27 00:29:41,205.205 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-27T00:29:41.275 INFO:teuthology.kill:2021-07-27 00:29:41,275.275 INFO:teuthology.orchestra.console:Power off smithi174
2021-07-27T00:29:41.303 INFO:teuthology.kill:2021-07-27 00:29:41,292.292 INFO:teuthology.orchestra.console:Power off smithi154
2021-07-27T00:29:41.432 INFO:teuthology.kill:2021-07-27 00:29:41,432.432 INFO:teuthology.orchestra.console:Power off for smithi174 completed
2021-07-27T00:29:41.453 INFO:teuthology.kill:2021-07-27 00:29:41,452.452 INFO:teuthology.orchestra.console:Power off for smithi154 completed
2021-07-27T00:29:41.579 INFO:teuthology.kill:2021-07-27 00:29:41,579.579 INFO:teuthology.lock.ops:unlocked smithi174.front.sepia.ceph.com
2021-07-27T00:29:41.598 INFO:teuthology.kill:2021-07-27 00:29:41,598.598 INFO:teuthology.lock.ops:unlocked smithi154.front.sepia.ceph.com
2021-07-27T00:31:41.916 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-07-27T00:31:42.007 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-07-27T00:31:42.008 INFO:teuthology.nuke:Checking targets against current locks
2021-07-27T00:31:42.085 INFO:teuthology.nuke:Not nuking smithi174.front.sepia.ceph.com because description doesn't match
2021-07-27T00:31:42.086 INFO:teuthology.nuke:Not nuking smithi154.front.sepia.ceph.com because description doesn't match