2021-05-01T06:31:09.654 INFO:root:teuthology version: 1.1.0-bea5b739
2021-05-01T06:31:09.758 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi033.front.sepia.ceph.com' reimaging
2021-05-01T06:31:09.758 INFO:teuthology.lock.ops:Updating [ubuntu@smithi033.front.sepia.ceph.com]: reset os type and version on server
2021-05-01T06:31:09.759 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server
2021-05-01T06:31:09.774 INFO:teuthology.lock.ops:Node 'ubuntu@smithi033.front.sepia.ceph.com' reimaging is complete
2021-05-01T06:31:09.775 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi172.front.sepia.ceph.com' reimaging
2021-05-01T06:31:09.775 INFO:teuthology.lock.ops:Updating [ubuntu@smithi172.front.sepia.ceph.com]: reset os type and version on server
2021-05-01T06:31:09.775 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server
2021-05-01T06:31:09.791 INFO:teuthology.lock.ops:Node 'ubuntu@smithi172.front.sepia.ceph.com' reimaging is complete
2021-05-01T06:31:10.006 INFO:teuthology.provision.fog.smithi033:Scheduling deploy of rhel 8.3
2021-05-01T06:31:10.008 INFO:teuthology.provision.fog.smithi172:Scheduling deploy of rhel 8.3
2021-05-01T06:31:10.300 INFO:teuthology.orchestra.console:Power off smithi172
2021-05-01T06:31:10.322 INFO:teuthology.orchestra.console:Power off smithi033
2021-05-01T06:31:18.719 INFO:teuthology.orchestra.console:Power off for smithi033 completed
2021-05-01T06:31:18.732 INFO:teuthology.orchestra.console:Power off for smithi172 completed
2021-05-01T06:31:18.820 INFO:teuthology.orchestra.console:Power on smithi033
2021-05-01T06:31:18.834 INFO:teuthology.orchestra.console:Power on smithi172
2021-05-01T06:31:27.193 INFO:teuthology.orchestra.console:Power on for smithi033 completed
2021-05-01T06:31:27.198 INFO:teuthology.orchestra.console:Power on for smithi172 completed
2021-05-01T06:31:27.294 INFO:teuthology.provision.fog.smithi033:Waiting for deploy to finish
2021-05-01T06:31:27.299 INFO:teuthology.provision.fog.smithi172:Waiting for deploy to finish
2021-05-01T06:35:34.768 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-05-01T06:35:35.109 INFO:teuthology.provision.fog.smithi033:Node is ready
2021-05-01T06:35:35.123 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033.front.sepia.ceph.com
2021-05-01T06:35:35.179 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33 smithi033.front.sepia.ceph.com smithi033
2021-05-01T06:35:35.514 INFO:teuthology.provision.fog.smithi033:Deploy complete!
2021-05-01T06:36:14.094 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-05-01T06:36:14.399 INFO:teuthology.provision.fog.smithi172:Node is ready
2021-05-01T06:36:14.414 INFO:teuthology.orchestra.run.smithi172.stdout:smithi172.front.sepia.ceph.com
2021-05-01T06:36:14.468 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172 smithi172.front.sepia.ceph.com smithi172
2021-05-01T06:36:14.802 INFO:teuthology.provision.fog.smithi172:Deploy complete!
2021-05-01T06:36:14.952 INFO:teuthology.lock.ops:Checking smithi033.front.sepia.ceph.com
2021-05-01T06:36:14.952 INFO:teuthology.lock.ops:New key found. Updating...
2021-05-01T06:36:14.969 INFO:teuthology.lock.ops:Checking smithi172.front.sepia.ceph.com
2021-05-01T06:36:14.970 INFO:teuthology.lock.ops:New key found. Updating...
2021-05-01T06:36:14.986 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: set os type and version on server
2021-05-01T06:36:14.986 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-05-01T06:36:15.285 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64
2021-05-01T06:36:15.340 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="Red Hat Enterprise Linux"
2021-05-01T06:36:15.340 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="8.3 (Ootpa)"
2021-05-01T06:36:15.340 INFO:teuthology.orchestra.run.smithi033.stdout:ID="rhel"
2021-05-01T06:36:15.341 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="fedora"
2021-05-01T06:36:15.341 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="8.3"
2021-05-01T06:36:15.341 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el8"
2021-05-01T06:36:15.341 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-05-01T06:36:15.341 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31"
2021-05-01T06:36:15.341 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-05-01T06:36:15.341 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://www.redhat.com/"
2021-05-01T06:36:15.341 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-05-01T06:36:15.341 INFO:teuthology.orchestra.run.smithi033.stdout:
2021-05-01T06:36:15.341 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-05-01T06:36:15.341 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-05-01T06:36:15.341 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-05-01T06:36:15.342 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-05-01T06:36:15.343 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server
2021-05-01T06:36:15.361 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: set os type and version on server
2021-05-01T06:36:15.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-05-01T06:36:15.639 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64
2021-05-01T06:36:15.694 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="Red Hat Enterprise Linux"
2021-05-01T06:36:15.694 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="8.3 (Ootpa)"
2021-05-01T06:36:15.694 INFO:teuthology.orchestra.run.smithi172.stdout:ID="rhel"
2021-05-01T06:36:15.695 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="fedora"
2021-05-01T06:36:15.695 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="8.3"
2021-05-01T06:36:15.695 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el8"
2021-05-01T06:36:15.695 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-05-01T06:36:15.695 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31"
2021-05-01T06:36:15.695 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-05-01T06:36:15.695 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://www.redhat.com/"
2021-05-01T06:36:15.695 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-05-01T06:36:15.695 INFO:teuthology.orchestra.run.smithi172.stdout:
2021-05-01T06:36:15.695 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-05-01T06:36:15.695 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-05-01T06:36:15.696 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-05-01T06:36:15.696 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-05-01T06:36:15.696 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server
2021-05-01T06:36:15.785 INFO:teuthology.dispatcher.supervisor:Running job 6082248
2021-05-01T06:36:15.785 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-04-29_13:53:41-rados-benhanokh-NCB-distro-basic-smithi/6082248 --name benhanokh-2021-04-29_13:53:41-rados-benhanokh-NCB-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/small-objects-balanced} -- /home/teuthworker/archive/benhanokh-2021-04-29_13:53:41-rados-benhanokh-NCB-distro-basic-smithi/6082248/orig.config.yaml
2021-05-01T06:36:15.790 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-04-29_13:53:41-rados-benhanokh-NCB-distro-basic-smithi/6082248
2021-05-01T06:36:15.791 INFO:teuthology.dispatcher.supervisor:Job PID: 14850
2021-05-01T06:36:15.791 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-05-01T09:54:29.570 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/benhanokh-2021-04-29_13:53:41-rados-benhanokh-NCB-distro-basic-smithi/jobs/ failed with status 500: <html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

2021-05-01T18:37:07.009 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-05-01T18:37:07.349 INFO:teuthology.kill:Killing Pids: {14850}
2021-05-01T18:37:08.419 INFO:teuthology.kill:Nuking machines: ['smithi033', 'smithi172']
2021-05-01T18:37:09.355 INFO:teuthology.kill:2021-05-01 18:37:09,354.354 INFO:teuthology.nuke:targets:
2021-05-01T18:37:09.355 INFO:teuthology.kill:  smithi033.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4BSxOBOlqcGvd7vJDWOH4xB+B6SZZbnEvKK58iCibuHGpHccc3OPAmEWkGHpfwir2dekg5iBu3z+HbrsSX+SjPS232v8rSxJw83ZzLw6NMCAA+ukHjlfzEXU03KH4hnWwShtG9PradY+zy7jOJ6coYBwbhgdfPmOSCIMe5Hqj7W3ef5zIv1grJXEuqwT/55FTNOMn0ukMdFeweRPen1GztUTu+XGIhXDX/eeK6d+2NQd2bHD3/iiHgpD04PKN7vNEHQFElGBS+U0Bf6eMhDz3TN/i4PKCjEwK6WYwhfgniRuq9xyUUMtXonNm2Dzfb/dLN9Jt+YJhVPfr41kH5ki9S1rV21TaekPY3vs9Xpk8n2RhMpEputN8nzXZx0hstnb3rsuKi4XHyPZe0uViyOpju9VjGNXs5tcxSa3Mw4gC3fgjnIcubIKVEbj0tZBBU4nN/z3ReJ9GecpytexWEGIenk2d7yBgI2yeBXqqlKKtVttudsO1xJ5NpPF9Ck6cmak=
2021-05-01T18:37:09.355 INFO:teuthology.kill:  smithi172.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmF29ji4bjbgUKLuGfoA4I0chmrOGDmVDx53AJqeiqzMKQfS62yR28VIRiK9mIs1pkJnpdX1++OdswM0nNtzj1oDX0M86DN1dvle1bHXsbSiq0jz9o+z431vEvFK3IiNueXxk55NYz9Jty4U2M0UiUASNiW6rxnH0ZPZ8fMZ9y4+fzQQAyMgKqcB9MgTDdJEn1026y7QY4H2ofQiKb173SWFYGlHwPr7FNfDtnTpL6XVErAvYdnO52ofZtmaaEpR5KkWIEg1Qj4P15iFfYfZlhteXrsSuv8F7DUF6fSXP1XwkYBhLXr95dCN/mHSLR8LKRgFFyOqx+qgP0y3ItWVztY0TLhBFjgfo4tEoJ04VNJNjJYB1lEHUHxhyVQi7mCOWyE9Pb4iesfGBE8IZSmrmiqf4iCie/8SuQkDiq5erHClp2NLyJbFETtkUTsi7zewYqcSm7vWmAAjkp2r7p1XF7EeQB/WTy6IGMGeeylhzcr7wpieZA/55GbnWhDurzRQM=
2021-05-01T18:37:09.355 INFO:teuthology.kill:2021-05-01 18:37:09,355.355 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-01T18:37:09.357 INFO:teuthology.kill:2021-05-01 18:37:09,357.357 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-01T18:37:09.419 INFO:teuthology.kill:2021-05-01 18:37:09,418.418 INFO:teuthology.orchestra.console:Power off smithi033
2021-05-01T18:37:09.432 INFO:teuthology.kill:2021-05-01 18:37:09,432.432 INFO:teuthology.orchestra.console:Power off smithi172
2021-05-01T18:37:21.906 INFO:teuthology.kill:2021-05-01 18:37:21,906.906 INFO:teuthology.orchestra.console:Power off for smithi033 completed
2021-05-01T18:37:21.911 INFO:teuthology.kill:2021-05-01 18:37:21,911.911 INFO:teuthology.orchestra.console:Power off for smithi172 completed
2021-05-01T18:37:22.141 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2021-05-01T18:37:22.141 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2021-05-01T18:37:22.141 INFO:teuthology.misc:Compressing logs...
2021-05-01T18:37:22.141 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-05-01T18:37:40.571 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/orchestra/remote.py", line 215, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_bea5b7399b9dcb9d756a32fdfc32f498319d425f/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-05-01T18:37:40.639 INFO:teuthology.kill:No teuthology processes running
2021-05-01T18:37:41.717 INFO:teuthology.kill:Nuking machines: ['smithi033', 'smithi172']
2021-05-01T18:37:42.626 INFO:teuthology.kill:2021-05-01 18:37:42,626.626 INFO:teuthology.nuke:targets:
2021-05-01T18:37:42.627 INFO:teuthology.kill:  smithi033.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4BSxOBOlqcGvd7vJDWOH4xB+B6SZZbnEvKK58iCibuHGpHccc3OPAmEWkGHpfwir2dekg5iBu3z+HbrsSX+SjPS232v8rSxJw83ZzLw6NMCAA+ukHjlfzEXU03KH4hnWwShtG9PradY+zy7jOJ6coYBwbhgdfPmOSCIMe5Hqj7W3ef5zIv1grJXEuqwT/55FTNOMn0ukMdFeweRPen1GztUTu+XGIhXDX/eeK6d+2NQd2bHD3/iiHgpD04PKN7vNEHQFElGBS+U0Bf6eMhDz3TN/i4PKCjEwK6WYwhfgniRuq9xyUUMtXonNm2Dzfb/dLN9Jt+YJhVPfr41kH5ki9S1rV21TaekPY3vs9Xpk8n2RhMpEputN8nzXZx0hstnb3rsuKi4XHyPZe0uViyOpju9VjGNXs5tcxSa3Mw4gC3fgjnIcubIKVEbj0tZBBU4nN/z3ReJ9GecpytexWEGIenk2d7yBgI2yeBXqqlKKtVttudsO1xJ5NpPF9Ck6cmak=
2021-05-01T18:37:42.627 INFO:teuthology.kill:  smithi172.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmF29ji4bjbgUKLuGfoA4I0chmrOGDmVDx53AJqeiqzMKQfS62yR28VIRiK9mIs1pkJnpdX1++OdswM0nNtzj1oDX0M86DN1dvle1bHXsbSiq0jz9o+z431vEvFK3IiNueXxk55NYz9Jty4U2M0UiUASNiW6rxnH0ZPZ8fMZ9y4+fzQQAyMgKqcB9MgTDdJEn1026y7QY4H2ofQiKb173SWFYGlHwPr7FNfDtnTpL6XVErAvYdnO52ofZtmaaEpR5KkWIEg1Qj4P15iFfYfZlhteXrsSuv8F7DUF6fSXP1XwkYBhLXr95dCN/mHSLR8LKRgFFyOqx+qgP0y3ItWVztY0TLhBFjgfo4tEoJ04VNJNjJYB1lEHUHxhyVQi7mCOWyE9Pb4iesfGBE8IZSmrmiqf4iCie/8SuQkDiq5erHClp2NLyJbFETtkUTsi7zewYqcSm7vWmAAjkp2r7p1XF7EeQB/WTy6IGMGeeylhzcr7wpieZA/55GbnWhDurzRQM=
2021-05-01T18:37:42.648 INFO:teuthology.kill:2021-05-01 18:37:42,648.648 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-01T18:37:42.650 INFO:teuthology.kill:2021-05-01 18:37:42,650.650 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-01T18:37:42.707 INFO:teuthology.kill:2021-05-01 18:37:42,707.707 INFO:teuthology.orchestra.console:Power off smithi172
2021-05-01T18:37:42.720 INFO:teuthology.kill:2021-05-01 18:37:42,720.720 INFO:teuthology.orchestra.console:Power off smithi033
2021-05-01T18:37:42.849 INFO:teuthology.kill:2021-05-01 18:37:42,848.848 INFO:teuthology.orchestra.console:Power off for smithi172 completed
2021-05-01T18:37:42.861 INFO:teuthology.kill:2021-05-01 18:37:42,861.861 INFO:teuthology.orchestra.console:Power off for smithi033 completed
2021-05-01T18:37:42.994 INFO:teuthology.kill:2021-05-01 18:37:42,993.993 INFO:teuthology.lock.ops:unlocked smithi172.front.sepia.ceph.com
2021-05-01T18:37:43.007 INFO:teuthology.kill:2021-05-01 18:37:43,007.007 INFO:teuthology.lock.ops:unlocked smithi033.front.sepia.ceph.com
2021-05-01T18:39:43.263 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-05-01T18:39:43.357 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-05-01T18:39:43.358 INFO:teuthology.nuke:Checking targets against current locks
2021-05-01T18:39:43.451 INFO:teuthology.nuke:Not nuking smithi033.front.sepia.ceph.com because description doesn't match
2021-05-01T18:39:43.451 INFO:teuthology.nuke:Not nuking smithi172.front.sepia.ceph.com because description doesn't match