2021-07-26T12:22:46.933 INFO:root:teuthology version: 1.1.0-b6f3efab
2021-07-26T12:22:47.021 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi061.front.sepia.ceph.com' reimaging
2021-07-26T12:22:47.023 INFO:teuthology.lock.ops:Updating [ubuntu@smithi061.front.sepia.ceph.com]: reset os type and version on server
2021-07-26T12:22:47.024 INFO:teuthology.lock.ops:Updating smithi061.front.sepia.ceph.com on lock server
2021-07-26T12:22:47.041 INFO:teuthology.lock.ops:Node 'ubuntu@smithi061.front.sepia.ceph.com' reimaging is complete
2021-07-26T12:22:47.041 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi084.front.sepia.ceph.com' reimaging
2021-07-26T12:22:47.042 INFO:teuthology.lock.ops:Updating [ubuntu@smithi084.front.sepia.ceph.com]: reset os type and version on server
2021-07-26T12:22:47.042 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server
2021-07-26T12:22:47.059 INFO:teuthology.lock.ops:Node 'ubuntu@smithi084.front.sepia.ceph.com' reimaging is complete
2021-07-26T12:22:47.261 INFO:teuthology.provision.fog.smithi061:Scheduling deploy of centos 8.2
2021-07-26T12:22:47.289 INFO:teuthology.provision.fog.smithi084:Scheduling deploy of centos 8.2
2021-07-26T12:22:47.643 INFO:teuthology.orchestra.console:Power off smithi061
2021-07-26T12:22:47.674 INFO:teuthology.orchestra.console:Power off smithi084
2021-07-26T12:22:47.782 INFO:teuthology.orchestra.console:Power off for smithi061 completed
2021-07-26T12:22:47.820 INFO:teuthology.orchestra.console:Power off for smithi084 completed
2021-07-26T12:22:47.885 INFO:teuthology.orchestra.console:Power on smithi061
2021-07-26T12:22:47.921 INFO:teuthology.orchestra.console:Power on smithi084
2021-07-26T12:22:52.168 INFO:teuthology.orchestra.console:Power on for smithi084 completed
2021-07-26T12:22:52.269 INFO:teuthology.provision.fog.smithi084:Waiting for deploy to finish
2021-07-26T12:22:56.303 INFO:teuthology.orchestra.console:Power on for smithi061 completed
2021-07-26T12:22:56.404 INFO:teuthology.provision.fog.smithi061:Waiting for deploy to finish
2021-07-26T12:27:34.719 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-26T12:27:35.058 INFO:teuthology.provision.fog.smithi061:Node is ready
2021-07-26T12:27:35.073 INFO:teuthology.orchestra.run.smithi061.stdout:smithi061.front.sepia.ceph.com
2021-07-26T12:27:35.129 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61 smithi061.front.sepia.ceph.com smithi061
2021-07-26T12:27:35.466 INFO:teuthology.provision.fog.smithi061:Deploy complete!
2021-07-26T12:27:36.898 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-26T12:27:37.201 INFO:teuthology.provision.fog.smithi084:Node is ready
2021-07-26T12:27:37.215 INFO:teuthology.orchestra.run.smithi084.stdout:smithi084.front.sepia.ceph.com
2021-07-26T12:27:37.269 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84 smithi084.front.sepia.ceph.com smithi084
2021-07-26T12:27:37.589 INFO:teuthology.provision.fog.smithi084:Deploy complete!
2021-07-26T12:27:37.706 INFO:teuthology.lock.ops:Checking smithi061.front.sepia.ceph.com
2021-07-26T12:27:37.707 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-26T12:27:37.722 INFO:teuthology.lock.ops:Checking smithi084.front.sepia.ceph.com
2021-07-26T12:27:37.722 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-26T12:27:37.736 INFO:teuthology.lock.ops:Updating [smithi061.front.sepia.ceph.com]: set os type and version on server
2021-07-26T12:27:37.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-26T12:27:38.016 INFO:teuthology.orchestra.run.smithi061.stdout:x86_64
2021-07-26T12:27:38.071 INFO:teuthology.orchestra.run.smithi061.stdout:NAME="CentOS Linux"
2021-07-26T12:27:38.071 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION="8 (Core)"
2021-07-26T12:27:38.072 INFO:teuthology.orchestra.run.smithi061.stdout:ID="centos"
2021-07-26T12:27:38.072 INFO:teuthology.orchestra.run.smithi061.stdout:ID_LIKE="rhel fedora"
2021-07-26T12:27:38.072 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION_ID="8"
2021-07-26T12:27:38.072 INFO:teuthology.orchestra.run.smithi061.stdout:PLATFORM_ID="platform:el8"
2021-07-26T12:27:38.073 INFO:teuthology.orchestra.run.smithi061.stdout:PRETTY_NAME="CentOS Linux 8 (Core)"
2021-07-26T12:27:38.073 INFO:teuthology.orchestra.run.smithi061.stdout:ANSI_COLOR="0;31"
2021-07-26T12:27:38.073 INFO:teuthology.orchestra.run.smithi061.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-07-26T12:27:38.073 INFO:teuthology.orchestra.run.smithi061.stdout:HOME_URL="https://www.centos.org/"
2021-07-26T12:27:38.073 INFO:teuthology.orchestra.run.smithi061.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-07-26T12:27:38.074 INFO:teuthology.orchestra.run.smithi061.stdout:
2021-07-26T12:27:38.074 INFO:teuthology.orchestra.run.smithi061.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-07-26T12:27:38.074 INFO:teuthology.orchestra.run.smithi061.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-07-26T12:27:38.074 INFO:teuthology.orchestra.run.smithi061.stdout:REDHAT_SUPPORT_PRODUCT="centos"
2021-07-26T12:27:38.075 INFO:teuthology.orchestra.run.smithi061.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8"
2021-07-26T12:27:38.075 INFO:teuthology.orchestra.run.smithi061.stdout:
2021-07-26T12:27:38.076 INFO:teuthology.lock.ops:Updating smithi061.front.sepia.ceph.com on lock server
2021-07-26T12:27:38.089 INFO:teuthology.lock.ops:Updating [smithi084.front.sepia.ceph.com]: set os type and version on server
2021-07-26T12:27:38.089 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-26T12:27:38.354 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64
2021-07-26T12:27:38.407 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Linux"
2021-07-26T12:27:38.407 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="8 (Core)"
2021-07-26T12:27:38.407 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos"
2021-07-26T12:27:38.407 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora"
2021-07-26T12:27:38.408 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="8"
2021-07-26T12:27:38.408 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el8"
2021-07-26T12:27:38.408 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Linux 8 (Core)"
2021-07-26T12:27:38.408 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31"
2021-07-26T12:27:38.409 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-07-26T12:27:38.409 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://www.centos.org/"
2021-07-26T12:27:38.409 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-07-26T12:27:38.409 INFO:teuthology.orchestra.run.smithi084.stdout:
2021-07-26T12:27:38.410 INFO:teuthology.orchestra.run.smithi084.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-07-26T12:27:38.410 INFO:teuthology.orchestra.run.smithi084.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-07-26T12:27:38.410 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="centos"
2021-07-26T12:27:38.410 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8"
2021-07-26T12:27:38.410 INFO:teuthology.orchestra.run.smithi084.stdout:
2021-07-26T12:27:38.411 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server
2021-07-26T12:27:38.485 INFO:teuthology.dispatcher.supervisor:Running job 6293457
2021-07-26T12:27:38.486 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-07-26_12:21:56-rados:cephadm:thrash-WIP_GBH_obj_scrubber2-distro-basic-smithi/6293457 --name benhanokh-2021-07-26_12:21:56-rados:cephadm:thrash-WIP_GBH_obj_scrubber2-distro-basic-smithi --description rados:cephadm:thrash/{0-distro/centos_8.2_kubic_stable 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} -- /home/teuthworker/archive/benhanokh-2021-07-26_12:21:56-rados:cephadm:thrash-WIP_GBH_obj_scrubber2-distro-basic-smithi/6293457/orig.config.yaml
2021-07-26T12:27:38.489 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-07-26_12:21:56-rados:cephadm:thrash-WIP_GBH_obj_scrubber2-distro-basic-smithi/6293457
2021-07-26T12:27:38.489 INFO:teuthology.dispatcher.supervisor:Job PID: 21683
2021-07-26T12:27:38.490 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-07-27T00:28:29.235 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-07-27T00:28:29.668 INFO:teuthology.kill:Killing Pids: {21683}
2021-07-27T00:28:31.036 INFO:teuthology.kill:Nuking machines: ['smithi061', 'smithi084']
2021-07-27T00:28:32.104 INFO:teuthology.kill:2021-07-27 00:28:32,104.104 INFO:teuthology.nuke:targets:
2021-07-27T00:28:32.105 INFO:teuthology.kill:  smithi061.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMKrd2FMvoYOQYWZ0Kapalh5DFXV327myWhi269nD0OX67j0AUmn2VHiA3xbK7Dd2Yq5EbyLIopzm+qDAflDQdA1K1ONU17u+otNeSlCVk4Xk+aOZrowUxWxj6zKO/veV7u25OXLO2Klta1NYRAv+bI+fbwA3BeO++yH1G7wi8rP4Tv6DIMIBOUJUPwnvd8S8+9d/yzhawg909IyK0S/qz+pt4lrcOltjomjjOHhEoTVJ6lJTAWUZ3oDfqos7kqfU9NaoJdIqiGv8Q9kWcx+cLVaWFtdlMxCYxRQxYnzV3vR5PgZ67LPBKkW+Kqmdyg4fad+joXWUQuSkMYS4cev8YWYGLYc7o2lVklLuj8VGo+QqZcupTWwsm158iOEhW6Ywuk1cfqRg0LnE/vVXp+Oyg5Ytji9YC24oXR7OIuTDXvbentu/rRKNUBAma/K2OoARJrPVpZ4mHCI6wFifN1mwuE+b1tF0jtFvdqG3ylCDYzAu4t+VVGOFbp85888NH6hs=
2021-07-27T00:28:32.106 INFO:teuthology.kill:  smithi084.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTRBjRX89RfjhMRLKKB0qpGsCN24JeBgUeKzwFUQv6fIYDWMtWd8tUmW40eG8/oHZ2ynPXUnF9dxzZXyt7JPtftp99SW1PDq94dRHSbHMvQUp45Yf6bcKi1YsqYB08mc8NkeWOsXkEfebHNU1sz5aq7gJW8/LHn5lK4OFkBWe6tfhAX8b/lX9I3C9jH2yB49QUo1jumZR2xe7T4qVFMvOVwpi+eJj3KT9+WTsWCeGb3fpB3hXzaR7xEMSYLfy3ZEuFVsEUIxAkdt6icC9d2lBQatULSYMqrkoSTA88lPp8muZ4P1QUuSJBK9k5BPNxXkYHX/1FyI2t6AJ3PxJFL91Mk3KN1VTBElei5G9FWRitIP10ok4qrz36ged6B4bJwsjmCrOOmSDsPccEvaEh4cQ6d/wSm2fvowKu3QMnBtPVXWJXQbOZ/0fRsLiYP5N8g8KvEGwNuz/BB9YP+HMtxSgBXtZtCx7POIeZqOZeORluJQHnverZhuA166iu+ENpz0M=
2021-07-27T00:28:32.106 INFO:teuthology.kill:2021-07-27 00:28:32,104.104 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-27T00:28:32.107 INFO:teuthology.kill:2021-07-27 00:28:32,107.107 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-27T00:28:32.191 INFO:teuthology.kill:2021-07-27 00:28:32,190.190 INFO:teuthology.orchestra.console:Power off smithi061
2021-07-27T00:28:32.213 INFO:teuthology.kill:2021-07-27 00:28:32,213.213 INFO:teuthology.orchestra.console:Power off smithi084
2021-07-27T00:28:40.589 INFO:teuthology.kill:2021-07-27 00:28:40,589.589 INFO:teuthology.orchestra.console:Power off for smithi061 completed
2021-07-27T00:28:40.606 INFO:teuthology.kill:2021-07-27 00:28:40,605.605 INFO:teuthology.orchestra.console:Power off for smithi084 completed
2021-07-27T00:28:40.864 INFO:teuthology.task.internal:roles: ubuntu@smithi061.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a']
2021-07-27T00:28:40.865 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a']
2021-07-27T00:28:40.865 INFO:teuthology.misc:Compressing logs...
2021-07-27T00:28:40.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-27T00:29:40.928 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/remote.py", line 509, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-07-27T00:29:40.994 INFO:teuthology.kill:No teuthology processes running
2021-07-27T00:29:42.333 INFO:teuthology.kill:Nuking machines: ['smithi061', 'smithi084']
2021-07-27T00:29:43.414 INFO:teuthology.kill:2021-07-27 00:29:43,414.414 INFO:teuthology.nuke:targets:
2021-07-27T00:29:43.415 INFO:teuthology.kill:  smithi061.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMKrd2FMvoYOQYWZ0Kapalh5DFXV327myWhi269nD0OX67j0AUmn2VHiA3xbK7Dd2Yq5EbyLIopzm+qDAflDQdA1K1ONU17u+otNeSlCVk4Xk+aOZrowUxWxj6zKO/veV7u25OXLO2Klta1NYRAv+bI+fbwA3BeO++yH1G7wi8rP4Tv6DIMIBOUJUPwnvd8S8+9d/yzhawg909IyK0S/qz+pt4lrcOltjomjjOHhEoTVJ6lJTAWUZ3oDfqos7kqfU9NaoJdIqiGv8Q9kWcx+cLVaWFtdlMxCYxRQxYnzV3vR5PgZ67LPBKkW+Kqmdyg4fad+joXWUQuSkMYS4cev8YWYGLYc7o2lVklLuj8VGo+QqZcupTWwsm158iOEhW6Ywuk1cfqRg0LnE/vVXp+Oyg5Ytji9YC24oXR7OIuTDXvbentu/rRKNUBAma/K2OoARJrPVpZ4mHCI6wFifN1mwuE+b1tF0jtFvdqG3ylCDYzAu4t+VVGOFbp85888NH6hs=
2021-07-27T00:29:43.415 INFO:teuthology.kill:  smithi084.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTRBjRX89RfjhMRLKKB0qpGsCN24JeBgUeKzwFUQv6fIYDWMtWd8tUmW40eG8/oHZ2ynPXUnF9dxzZXyt7JPtftp99SW1PDq94dRHSbHMvQUp45Yf6bcKi1YsqYB08mc8NkeWOsXkEfebHNU1sz5aq7gJW8/LHn5lK4OFkBWe6tfhAX8b/lX9I3C9jH2yB49QUo1jumZR2xe7T4qVFMvOVwpi+eJj3KT9+WTsWCeGb3fpB3hXzaR7xEMSYLfy3ZEuFVsEUIxAkdt6icC9d2lBQatULSYMqrkoSTA88lPp8muZ4P1QUuSJBK9k5BPNxXkYHX/1FyI2t6AJ3PxJFL91Mk3KN1VTBElei5G9FWRitIP10ok4qrz36ged6B4bJwsjmCrOOmSDsPccEvaEh4cQ6d/wSm2fvowKu3QMnBtPVXWJXQbOZ/0fRsLiYP5N8g8KvEGwNuz/BB9YP+HMtxSgBXtZtCx7POIeZqOZeORluJQHnverZhuA166iu+ENpz0M=
2021-07-27T00:29:43.435 INFO:teuthology.kill:2021-07-27 00:29:43,434.434 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-27T00:29:43.441 INFO:teuthology.kill:2021-07-27 00:29:43,440.440 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-27T00:29:43.502 INFO:teuthology.kill:2021-07-27 00:29:43,502.502 INFO:teuthology.orchestra.console:Power off smithi084
2021-07-27T00:29:43.523 INFO:teuthology.kill:2021-07-27 00:29:43,523.523 INFO:teuthology.orchestra.console:Power off smithi061
2021-07-27T00:29:43.654 INFO:teuthology.kill:2021-07-27 00:29:43,653.653 INFO:teuthology.orchestra.console:Power off for smithi084 completed
2021-07-27T00:29:43.665 INFO:teuthology.kill:2021-07-27 00:29:43,665.665 INFO:teuthology.orchestra.console:Power off for smithi061 completed
2021-07-27T00:29:43.799 INFO:teuthology.kill:2021-07-27 00:29:43,797.797 INFO:teuthology.lock.ops:unlocked smithi084.front.sepia.ceph.com
2021-07-27T00:29:43.809 INFO:teuthology.kill:2021-07-27 00:29:43,809.809 INFO:teuthology.lock.ops:unlocked smithi061.front.sepia.ceph.com
2021-07-27T00:31:44.135 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-07-27T00:31:44.218 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-07-27T00:31:44.219 INFO:teuthology.nuke:Checking targets against current locks
2021-07-27T00:31:44.290 INFO:teuthology.nuke:Not nuking smithi061.front.sepia.ceph.com because description doesn't match
2021-07-27T00:31:44.290 INFO:teuthology.nuke:Not nuking smithi084.front.sepia.ceph.com because description doesn't match