2021-06-15T04:17:41.161 INFO:root:teuthology version: 1.1.0-f359b10d
2021-06-15T04:17:41.258 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi029.front.sepia.ceph.com' reimaging
2021-06-15T04:17:41.259 INFO:teuthology.lock.ops:Updating [ubuntu@smithi029.front.sepia.ceph.com]: reset os type and version on server
2021-06-15T04:17:41.259 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server
2021-06-15T04:17:41.273 INFO:teuthology.lock.ops:Node 'ubuntu@smithi029.front.sepia.ceph.com' reimaging is complete
2021-06-15T04:17:41.274 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging
2021-06-15T04:17:41.274 INFO:teuthology.lock.ops:Updating [ubuntu@smithi146.front.sepia.ceph.com]: reset os type and version on server
2021-06-15T04:17:41.274 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2021-06-15T04:17:41.287 INFO:teuthology.lock.ops:Node 'ubuntu@smithi146.front.sepia.ceph.com' reimaging is complete
2021-06-15T04:17:41.471 INFO:teuthology.provision.fog.smithi029:Scheduling deploy of rhel 8.3
2021-06-15T04:17:41.503 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of rhel 8.3
2021-06-15T04:17:41.917 INFO:teuthology.orchestra.console:Power off smithi029
2021-06-15T04:17:41.931 INFO:teuthology.orchestra.console:Power off smithi146
2021-06-15T04:17:50.290 INFO:teuthology.orchestra.console:Power off for smithi029 completed
2021-06-15T04:17:50.295 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2021-06-15T04:17:50.392 INFO:teuthology.orchestra.console:Power on smithi029
2021-06-15T04:17:50.400 INFO:teuthology.orchestra.console:Power on smithi146
2021-06-15T04:17:54.654 INFO:teuthology.orchestra.console:Power on for smithi146 completed
2021-06-15T04:17:54.755 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish
2021-06-15T04:17:58.759 INFO:teuthology.orchestra.console:Power on for smithi029 completed
2021-06-15T04:17:58.860 INFO:teuthology.provision.fog.smithi029:Waiting for deploy to finish
2021-06-15T04:22:59.693 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-06-15T04:23:05.096 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-06-15T04:23:05.467 INFO:teuthology.provision.fog.smithi029:Node is ready
2021-06-15T04:23:05.482 INFO:teuthology.orchestra.run.smithi029.stdout:smithi029.front.sepia.ceph.com
2021-06-15T04:23:05.538 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29 smithi029.front.sepia.ceph.com smithi029
2021-06-15T04:23:05.909 INFO:teuthology.provision.fog.smithi029:Deploy complete!
2021-06-15T04:23:20.110 INFO:teuthology.provision.fog.smithi146:Node is ready
2021-06-15T04:23:20.169 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com
2021-06-15T04:23:20.225 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146
2021-06-15T04:23:20.587 INFO:teuthology.provision.fog.smithi146:Deploy complete!
2021-06-15T04:23:20.700 INFO:teuthology.lock.ops:Checking smithi029.front.sepia.ceph.com
2021-06-15T04:23:20.701 INFO:teuthology.lock.ops:New key found. Updating...
2021-06-15T04:23:20.715 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com
2021-06-15T04:23:20.715 INFO:teuthology.lock.ops:New key found. Updating...
2021-06-15T04:23:20.728 INFO:teuthology.lock.ops:Updating [smithi029.front.sepia.ceph.com]: set os type and version on server
2021-06-15T04:23:20.728 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-06-15T04:23:21.023 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64
2021-06-15T04:23:21.078 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="Red Hat Enterprise Linux"
2021-06-15T04:23:21.078 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="8.3 (Ootpa)"
2021-06-15T04:23:21.079 INFO:teuthology.orchestra.run.smithi029.stdout:ID="rhel"
2021-06-15T04:23:21.079 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="fedora"
2021-06-15T04:23:21.079 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="8.3"
2021-06-15T04:23:21.079 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el8"
2021-06-15T04:23:21.080 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-06-15T04:23:21.080 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31"
2021-06-15T04:23:21.080 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-06-15T04:23:21.080 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://www.redhat.com/"
2021-06-15T04:23:21.081 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-06-15T04:23:21.081 INFO:teuthology.orchestra.run.smithi029.stdout:
2021-06-15T04:23:21.081 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-06-15T04:23:21.081 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-06-15T04:23:21.081 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-06-15T04:23:21.082 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-06-15T04:23:21.083 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server
2021-06-15T04:23:21.098 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server
2021-06-15T04:23:21.098 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-06-15T04:23:21.320 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64
2021-06-15T04:23:21.375 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="Red Hat Enterprise Linux"
2021-06-15T04:23:21.376 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8.3 (Ootpa)"
2021-06-15T04:23:21.376 INFO:teuthology.orchestra.run.smithi146.stdout:ID="rhel"
2021-06-15T04:23:21.376 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="fedora"
2021-06-15T04:23:21.376 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8.3"
2021-06-15T04:23:21.377 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8"
2021-06-15T04:23:21.377 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-06-15T04:23:21.377 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31"
2021-06-15T04:23:21.377 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-06-15T04:23:21.378 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://www.redhat.com/"
2021-06-15T04:23:21.378 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-06-15T04:23:21.378 INFO:teuthology.orchestra.run.smithi146.stdout:
2021-06-15T04:23:21.378 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-06-15T04:23:21.378 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-06-15T04:23:21.379 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-06-15T04:23:21.379 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-06-15T04:23:21.379 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2021-06-15T04:23:21.458 INFO:teuthology.dispatcher.supervisor:Running job 6173101
2021-06-15T04:23:21.458 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-06-15_04:17:01-fs-pacific-distro-basic-smithi/6173101 --name teuthology-2021-06-15_04:17:01-fs-pacific-distro-basic-smithi --description fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/fs/snaps}} -- /home/teuthworker/archive/teuthology-2021-06-15_04:17:01-fs-pacific-distro-basic-smithi/6173101/orig.config.yaml
2021-06-15T04:23:21.462 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-06-15_04:17:01-fs-pacific-distro-basic-smithi/6173101
2021-06-15T04:23:21.463 INFO:teuthology.dispatcher.supervisor:Job PID: 19458
2021-06-15T04:23:21.463 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-06-15T16:24:07.986 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-06-15T16:24:08.183 INFO:teuthology.kill:Killing Pids: {19458}
2021-06-15T16:24:09.188 INFO:teuthology.kill:Nuking machines: ['smithi029', 'smithi146']
2021-06-15T16:24:10.063 INFO:teuthology.kill:2021-06-15 16:24:10,062.062 INFO:teuthology.nuke:targets:
2021-06-15T16:24:10.063 INFO:teuthology.kill:  smithi029.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGx6e+S+cFrWLE0CLd2Bv7c+W5xqC9F4/Kj+wdKoK3jL9DaK66jyNimrU2k3J86cCcEIotqw7D2wVho3/7UHQ35tPzhj9Fu9mFLCIKX46BPXMZ/0S9t42WWxMa6Z35HGVUeOqgMW2hRC++B1Oexz8sIIHZ0hDc+956Da8AvkC9p8i/YPjMutw/3CWC02p+hmsrom92Dkp/yT1fq6QFpNxpUk7pC+RLr2k2J9+1LwtkUi3jvyyuC+AgTlpzVdnWvcV58ufeBfZrLFXPD4uNB6F1TCKz0F3kFa/MDch8W1mfAEVvijfTwSgu0GZk7IPNzcrYDgbqP+pMYKaqqCl4lG7hSUqCs/YOOjQBQz/quGgW9a9Ci5xp6j/xY8UO9OogE4ehhp73I84Hu6OJx7YV75brQhM/09rL5lT/AxzdO+jKUN92U/crZiXr8hfOMnEerv3hZq2A+cGEy2cHFtnnt/0DpRAXHuQPjq1i+j0OM64GVg2ws1dU3OEymnSUiIKGR2E=
2021-06-15T16:24:10.064 INFO:teuthology.kill:  smithi146.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAZaMuN22vp46yFDrss17cpzfEe5I1A4Rft7msEG6EkEINfRhLq8HFraubbVWF1tELCBXnjaKjxrIkTkO8c1Jfz4OBlLLIk/SwwuhsLbnrL/mF1euPNZlIhKyY7ipOwKihB/gujSPvKXhm6O8P94MWtVVEmHG5ZIlbk15jCpWwPp4OcFTZT9mb/dwhAupXUMnL8MnBRxbB1uKF3SH3fX09mYSyaDtlxPUJEQFvpDQhJtnwuQJ/x4bEoFLVcWyqb34Ugy4lN0+NOpRX8lGJI9tJN6cXo6wNPsBs6VPhbUfpe1b5tb0JVhUgmn64dWRDlNxN3nyaJWjZdJMdoKJxupL2L4JjCNjpGpXTP9eOmqwQToARwgAOh+oGOL+hK3cMdLbm5MqVTyveFLgicaeANpM2I/OzwQ+nMOg3C5keL4FpkLJ6lou8JvxgcqevpOS10fbdUIWvFG5jE2VQlOxvocQghhMA8E95BLKm4ri/2scb+HHsydPCYeRsg7uTusv7OCE=
2021-06-15T16:24:10.064 INFO:teuthology.kill:2021-06-15 16:24:10,063.063 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-06-15T16:24:10.065 INFO:teuthology.kill:2021-06-15 16:24:10,065.065 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-06-15T16:24:10.123 INFO:teuthology.kill:2021-06-15 16:24:10,123.123 INFO:teuthology.orchestra.console:Power off smithi029
2021-06-15T16:24:10.135 INFO:teuthology.kill:2021-06-15 16:24:10,134.134 INFO:teuthology.orchestra.console:Power off smithi146
2021-06-15T16:24:22.602 INFO:teuthology.kill:2021-06-15 16:24:22,602.602 INFO:teuthology.orchestra.console:Power off for smithi029 completed
2021-06-15T16:24:22.608 INFO:teuthology.kill:2021-06-15 16:24:22,607.607 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2021-06-15T16:24:22.829 INFO:teuthology.task.internal:roles: ubuntu@smithi029.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2021-06-15T16:24:22.829 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2021-06-15T16:24:22.830 INFO:teuthology.misc:Compressing logs...
2021-06-15T16:24:22.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-06-15T16:24:57.295 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/remote.py", line 509, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-06-15T16:24:57.365 INFO:teuthology.kill:No teuthology processes running
2021-06-15T16:24:58.420 INFO:teuthology.kill:Nuking machines: ['smithi029', 'smithi146']
2021-06-15T16:24:59.301 INFO:teuthology.kill:2021-06-15 16:24:59,301.301 INFO:teuthology.nuke:targets:
2021-06-15T16:24:59.302 INFO:teuthology.kill:  smithi029.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGx6e+S+cFrWLE0CLd2Bv7c+W5xqC9F4/Kj+wdKoK3jL9DaK66jyNimrU2k3J86cCcEIotqw7D2wVho3/7UHQ35tPzhj9Fu9mFLCIKX46BPXMZ/0S9t42WWxMa6Z35HGVUeOqgMW2hRC++B1Oexz8sIIHZ0hDc+956Da8AvkC9p8i/YPjMutw/3CWC02p+hmsrom92Dkp/yT1fq6QFpNxpUk7pC+RLr2k2J9+1LwtkUi3jvyyuC+AgTlpzVdnWvcV58ufeBfZrLFXPD4uNB6F1TCKz0F3kFa/MDch8W1mfAEVvijfTwSgu0GZk7IPNzcrYDgbqP+pMYKaqqCl4lG7hSUqCs/YOOjQBQz/quGgW9a9Ci5xp6j/xY8UO9OogE4ehhp73I84Hu6OJx7YV75brQhM/09rL5lT/AxzdO+jKUN92U/crZiXr8hfOMnEerv3hZq2A+cGEy2cHFtnnt/0DpRAXHuQPjq1i+j0OM64GVg2ws1dU3OEymnSUiIKGR2E=
2021-06-15T16:24:59.302 INFO:teuthology.kill:  smithi146.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAZaMuN22vp46yFDrss17cpzfEe5I1A4Rft7msEG6EkEINfRhLq8HFraubbVWF1tELCBXnjaKjxrIkTkO8c1Jfz4OBlLLIk/SwwuhsLbnrL/mF1euPNZlIhKyY7ipOwKihB/gujSPvKXhm6O8P94MWtVVEmHG5ZIlbk15jCpWwPp4OcFTZT9mb/dwhAupXUMnL8MnBRxbB1uKF3SH3fX09mYSyaDtlxPUJEQFvpDQhJtnwuQJ/x4bEoFLVcWyqb34Ugy4lN0+NOpRX8lGJI9tJN6cXo6wNPsBs6VPhbUfpe1b5tb0JVhUgmn64dWRDlNxN3nyaJWjZdJMdoKJxupL2L4JjCNjpGpXTP9eOmqwQToARwgAOh+oGOL+hK3cMdLbm5MqVTyveFLgicaeANpM2I/OzwQ+nMOg3C5keL4FpkLJ6lou8JvxgcqevpOS10fbdUIWvFG5jE2VQlOxvocQghhMA8E95BLKm4ri/2scb+HHsydPCYeRsg7uTusv7OCE=
2021-06-15T16:24:59.321 INFO:teuthology.kill:2021-06-15 16:24:59,321.321 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-06-15T16:24:59.323 INFO:teuthology.kill:2021-06-15 16:24:59,322.322 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-06-15T16:24:59.376 INFO:teuthology.kill:2021-06-15 16:24:59,376.376 INFO:teuthology.orchestra.console:Power off smithi146
2021-06-15T16:24:59.389 INFO:teuthology.kill:2021-06-15 16:24:59,389.389 INFO:teuthology.orchestra.console:Power off smithi029
2021-06-15T16:24:59.516 INFO:teuthology.kill:2021-06-15 16:24:59,515.515 INFO:teuthology.orchestra.console:Power off for smithi146 completed
2021-06-15T16:24:59.525 INFO:teuthology.kill:2021-06-15 16:24:59,525.525 INFO:teuthology.orchestra.console:Power off for smithi029 completed
2021-06-15T16:24:59.668 INFO:teuthology.kill:2021-06-15 16:24:59,667.667 INFO:teuthology.lock.ops:unlocked smithi029.front.sepia.ceph.com
2021-06-15T16:24:59.674 INFO:teuthology.kill:2021-06-15 16:24:59,674.674 INFO:teuthology.lock.ops:unlocked smithi146.front.sepia.ceph.com
2021-06-15T16:26:59.933 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15