2021-07-27T22:04:02.305 INFO:root:teuthology version: 1.1.0-73aa7e3b
2021-07-27T22:04:02.398 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging
2021-07-27T22:04:02.399 INFO:teuthology.lock.ops:Updating [ubuntu@smithi110.front.sepia.ceph.com]: reset os type and version on server
2021-07-27T22:04:02.399 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2021-07-27T22:04:02.413 INFO:teuthology.lock.ops:Node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging is complete
2021-07-27T22:04:02.414 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi143.front.sepia.ceph.com' reimaging
2021-07-27T22:04:02.414 INFO:teuthology.lock.ops:Updating [ubuntu@smithi143.front.sepia.ceph.com]: reset os type and version on server
2021-07-27T22:04:02.414 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server
2021-07-27T22:04:02.428 INFO:teuthology.lock.ops:Node 'ubuntu@smithi143.front.sepia.ceph.com' reimaging is complete
2021-07-27T22:04:02.631 INFO:teuthology.provision.fog.smithi143:Scheduling deploy of rhel 8.4
2021-07-27T22:04:02.669 INFO:teuthology.provision.fog.smithi110:Scheduling deploy of rhel 8.4
2021-07-27T22:04:03.017 INFO:teuthology.orchestra.console:Power off smithi143
2021-07-27T22:04:03.059 INFO:teuthology.orchestra.console:Power off smithi110
2021-07-27T22:04:11.395 INFO:teuthology.orchestra.console:Power off for smithi143 completed
2021-07-27T22:04:11.496 INFO:teuthology.orchestra.console:Power on smithi143
2021-07-27T22:04:15.529 INFO:teuthology.orchestra.console:Power off for smithi110 completed
2021-07-27T22:04:15.631 INFO:teuthology.orchestra.console:Power on smithi110
2021-07-27T22:04:19.886 INFO:teuthology.orchestra.console:Power on for smithi110 completed
2021-07-27T22:04:19.894 INFO:teuthology.orchestra.console:Power on for smithi143 completed
2021-07-27T22:04:19.988 INFO:teuthology.provision.fog.smithi110:Waiting for deploy to finish
2021-07-27T22:04:19.995 INFO:teuthology.provision.fog.smithi143:Waiting for deploy to finish
2021-07-27T22:08:20.493 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-27T22:08:36.021 INFO:teuthology.provision.fog.smithi143:Node is ready
2021-07-27T22:08:36.040 INFO:teuthology.orchestra.run.smithi143.stdout:smithi204
2021-07-27T22:08:36.096 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.204 smithi204.front.sepia.ceph.com smithi204
2021-07-27T22:08:36.460 INFO:teuthology.provision.fog.smithi143:Deploy complete!
2021-07-27T22:08:58.203 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-27T22:08:58.429 INFO:teuthology.provision.fog.smithi110:Node is ready
2021-07-27T22:08:58.444 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110.front.sepia.ceph.com
2021-07-27T22:08:58.499 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110 smithi110.front.sepia.ceph.com smithi110
2021-07-27T22:08:58.833 INFO:teuthology.provision.fog.smithi110:Deploy complete!
2021-07-27T22:08:58.966 INFO:teuthology.lock.ops:Checking smithi110.front.sepia.ceph.com
2021-07-27T22:08:58.967 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-27T22:08:58.983 INFO:teuthology.lock.ops:Checking smithi143.front.sepia.ceph.com
2021-07-27T22:08:58.984 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-27T22:08:58.999 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: set os type and version on server
2021-07-27T22:08:59.000 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-27T22:08:59.227 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64
2021-07-27T22:08:59.281 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="Red Hat Enterprise Linux"
2021-07-27T22:08:59.281 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="8.4 (Ootpa)"
2021-07-27T22:08:59.282 INFO:teuthology.orchestra.run.smithi110.stdout:ID="rhel"
2021-07-27T22:08:59.282 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="fedora"
2021-07-27T22:08:59.282 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="8.4"
2021-07-27T22:08:59.282 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el8"
2021-07-27T22:08:59.283 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2021-07-27T22:08:59.283 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31"
2021-07-27T22:08:59.283 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2021-07-27T22:08:59.283 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://www.redhat.com/"
2021-07-27T22:08:59.284 INFO:teuthology.orchestra.run.smithi110.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2021-07-27T22:08:59.284 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-07-27T22:08:59.284 INFO:teuthology.orchestra.run.smithi110.stdout:
2021-07-27T22:08:59.284 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-07-27T22:08:59.284 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2021-07-27T22:08:59.285 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-07-27T22:08:59.285 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2021-07-27T22:08:59.286 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2021-07-27T22:08:59.302 INFO:teuthology.lock.ops:Updating [smithi143.front.sepia.ceph.com]: set os type and version on server
2021-07-27T22:08:59.302 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-27T22:08:59.538 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64
2021-07-27T22:08:59.594 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="Red Hat Enterprise Linux"
2021-07-27T22:08:59.595 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="8.4 (Ootpa)"
2021-07-27T22:08:59.595 INFO:teuthology.orchestra.run.smithi143.stdout:ID="rhel"
2021-07-27T22:08:59.595 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="fedora"
2021-07-27T22:08:59.595 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="8.4"
2021-07-27T22:08:59.596 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el8"
2021-07-27T22:08:59.596 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2021-07-27T22:08:59.596 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31"
2021-07-27T22:08:59.597 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2021-07-27T22:08:59.597 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://www.redhat.com/"
2021-07-27T22:08:59.597 INFO:teuthology.orchestra.run.smithi143.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2021-07-27T22:08:59.598 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-07-27T22:08:59.598 INFO:teuthology.orchestra.run.smithi143.stdout:
2021-07-27T22:08:59.598 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-07-27T22:08:59.598 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2021-07-27T22:08:59.599 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-07-27T22:08:59.599 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2021-07-27T22:08:59.600 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server
2021-07-27T22:08:59.682 INFO:teuthology.dispatcher.supervisor:Running job 6296411
2021-07-27T22:08:59.683 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-07-27_13:23:42-rados-wip_gbenhano_ncb_fix2-distro-basic-smithi/6296411 --name benhanokh-2021-07-27_13:23:42-rados-wip_gbenhano_ncb_fix2-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/cache-agent-small} -- /home/teuthworker/archive/benhanokh-2021-07-27_13:23:42-rados-wip_gbenhano_ncb_fix2-distro-basic-smithi/6296411/orig.config.yaml
2021-07-27T22:08:59.686 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-07-27_13:23:42-rados-wip_gbenhano_ncb_fix2-distro-basic-smithi/6296411
2021-07-27T22:08:59.687 INFO:teuthology.dispatcher.supervisor:Job PID: 24842
2021-07-27T22:08:59.687 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-07-28T10:09:47.684 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-07-28T10:09:47.912 INFO:teuthology.kill:Killing Pids: {24842}
2021-07-28T10:09:49.032 INFO:teuthology.kill:Nuking machines: ['smithi110', 'smithi143']
2021-07-28T10:09:49.952 INFO:teuthology.kill:2021-07-28 10:09:49,952.952 INFO:teuthology.nuke:targets:
2021-07-28T10:09:49.953 INFO:teuthology.kill:  smithi110.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJM1X7OKtAt+UPUaFFAioaJ+jexOM5q+oSKoKc+P/5sl/BYCQygZMfadYJ4dWUYYRolr5nK2SVeljawqYkiAdsJ/GUZHi0dsCJAanu8HPGcPl9MXYfICkveI4Mn6A2uOBkKuZWWOWNBpVsgKfEMK9lNxpJheRKN6I9R4ufc4J+wu5z1yLvA/eNVDCjp+UNpqsqg0rbAnI6c4scqItL0a2WqPQi9BUiFOEbnWHqOaqP2N2Sd+uA1BIO3/4mQYqsBo9JzTkWJLiSsLiwh+yoHbAPxWYURbdj3u39xSLkIIIfvFMxMvgxoBluuuphpBHPI942yxCStiLQVUR/g7WewmqO4MxiTys1zTRXM2IhB7PaIlTqyLlSnj2G3D1C41/QlJsgUhYAIOBOaJttTjVHnAbKzgYh3T8qPd19GbopEMUnRNTcFFYsuJRSVtklXDBlJRkiwZExdO056wBG2OjmVyFaXqrK3Q/jCWZCBYY9l48DrFP3yhIniRfcxgHgk6fwbHM=
2021-07-28T10:09:49.954 INFO:teuthology.kill:  smithi143.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6f3jB3jOPT/jovjuUjM/62FRPVmUryGKDxtQqCPVSToQO3+jQbvqDygYWl9ZMzcSnfrUEDekNvxvZ48D63ZyLt1Rya9MDnuTczG4I0nFKk8ix/DPBel7jxbCqkmzThWQKOsIIDbMCHXvx1vhOi9/Z594mOnCxGjJXu+Mz5+SRMBTx4uZBEcICpPbWyjIsyjjGNWyqz7EWn//wm+s44isXVo5zHTt5Fju9lI6KGe/goTzZidaJuzx75Sk2e/30R+2NO/wlz/CQZ4JITggQBKuLoKY1jYuYWfcbF7Kc7kiPTNvQyosSb9HThSYvASNQwkJ33ENye7VDgPxn5ewMhMLmBhA4jpQ2VFR3A8CoaUHyhIuiyj4K2QwzyvDNzs3FueWBxn9N1gvaix2QosyQYUjxCjGJTlCml/T+6haB0Hg07edO6DZNbq2jJOTnp3moktoLm3jATJrOoRTu/50f4ilQjynaMLuEizfRGGlUQw6kp488KLRkW97Oc46ynXaNAZc=
2021-07-28T10:09:49.954 INFO:teuthology.kill:2021-07-28 10:09:49,952.952 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-28T10:09:49.955 INFO:teuthology.kill:2021-07-28 10:09:49,955.955 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-28T10:09:50.012 INFO:teuthology.kill:2021-07-28 10:09:50,012.012 INFO:teuthology.orchestra.console:Power off smithi110
2021-07-28T10:09:50.025 INFO:teuthology.kill:2021-07-28 10:09:50,025.025 INFO:teuthology.orchestra.console:Power off smithi143
2021-07-28T10:09:58.389 INFO:teuthology.kill:2021-07-28 10:09:58,388.388 INFO:teuthology.orchestra.console:Power off for smithi110 completed
2021-07-28T10:09:58.397 INFO:teuthology.kill:2021-07-28 10:09:58,397.397 INFO:teuthology.orchestra.console:Power off for smithi143 completed
2021-07-28T10:09:58.654 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2021-07-28T10:09:58.655 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2021-07-28T10:09:58.656 INFO:teuthology.misc:Compressing logs...
2021-07-28T10:09:58.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-28T10:10:58.719 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/remote.py", line 509, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-07-28T10:10:58.794 INFO:teuthology.kill:No teuthology processes running
2021-07-28T10:10:59.921 INFO:teuthology.kill:Nuking machines: ['smithi110', 'smithi143']
2021-07-28T10:11:00.865 INFO:teuthology.kill:2021-07-28 10:11:00,864.864 INFO:teuthology.nuke:targets:
2021-07-28T10:11:00.866 INFO:teuthology.kill:  smithi110.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJM1X7OKtAt+UPUaFFAioaJ+jexOM5q+oSKoKc+P/5sl/BYCQygZMfadYJ4dWUYYRolr5nK2SVeljawqYkiAdsJ/GUZHi0dsCJAanu8HPGcPl9MXYfICkveI4Mn6A2uOBkKuZWWOWNBpVsgKfEMK9lNxpJheRKN6I9R4ufc4J+wu5z1yLvA/eNVDCjp+UNpqsqg0rbAnI6c4scqItL0a2WqPQi9BUiFOEbnWHqOaqP2N2Sd+uA1BIO3/4mQYqsBo9JzTkWJLiSsLiwh+yoHbAPxWYURbdj3u39xSLkIIIfvFMxMvgxoBluuuphpBHPI942yxCStiLQVUR/g7WewmqO4MxiTys1zTRXM2IhB7PaIlTqyLlSnj2G3D1C41/QlJsgUhYAIOBOaJttTjVHnAbKzgYh3T8qPd19GbopEMUnRNTcFFYsuJRSVtklXDBlJRkiwZExdO056wBG2OjmVyFaXqrK3Q/jCWZCBYY9l48DrFP3yhIniRfcxgHgk6fwbHM=
2021-07-28T10:11:00.866 INFO:teuthology.kill:  smithi143.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6f3jB3jOPT/jovjuUjM/62FRPVmUryGKDxtQqCPVSToQO3+jQbvqDygYWl9ZMzcSnfrUEDekNvxvZ48D63ZyLt1Rya9MDnuTczG4I0nFKk8ix/DPBel7jxbCqkmzThWQKOsIIDbMCHXvx1vhOi9/Z594mOnCxGjJXu+Mz5+SRMBTx4uZBEcICpPbWyjIsyjjGNWyqz7EWn//wm+s44isXVo5zHTt5Fju9lI6KGe/goTzZidaJuzx75Sk2e/30R+2NO/wlz/CQZ4JITggQBKuLoKY1jYuYWfcbF7Kc7kiPTNvQyosSb9HThSYvASNQwkJ33ENye7VDgPxn5ewMhMLmBhA4jpQ2VFR3A8CoaUHyhIuiyj4K2QwzyvDNzs3FueWBxn9N1gvaix2QosyQYUjxCjGJTlCml/T+6haB0Hg07edO6DZNbq2jJOTnp3moktoLm3jATJrOoRTu/50f4ilQjynaMLuEizfRGGlUQw6kp488KLRkW97Oc46ynXaNAZc=
2021-07-28T10:11:00.885 INFO:teuthology.kill:2021-07-28 10:11:00,885.885 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-28T10:11:00.888 INFO:teuthology.kill:2021-07-28 10:11:00,888.888 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-28T10:11:00.952 INFO:teuthology.kill:2021-07-28 10:11:00,951.951 INFO:teuthology.orchestra.console:Power off smithi110
2021-07-28T10:11:00.968 INFO:teuthology.kill:2021-07-28 10:11:00,967.967 INFO:teuthology.orchestra.console:Power off smithi143
2021-07-28T10:11:01.099 INFO:teuthology.kill:2021-07-28 10:11:01,098.098 INFO:teuthology.orchestra.console:Power off for smithi110 completed
2021-07-28T10:11:01.111 INFO:teuthology.kill:2021-07-28 10:11:01,111.111 INFO:teuthology.orchestra.console:Power off for smithi143 completed
2021-07-28T10:11:01.238 INFO:teuthology.kill:2021-07-28 10:11:01,238.238 INFO:teuthology.lock.ops:unlocked smithi110.front.sepia.ceph.com
2021-07-28T10:11:01.249 INFO:teuthology.kill:2021-07-28 10:11:01,249.249 INFO:teuthology.lock.ops:unlocked smithi143.front.sepia.ceph.com
2021-07-28T10:13:01.572 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-07-28T10:13:01.664 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-07-28T10:13:01.664 INFO:teuthology.nuke:Checking targets against current locks
2021-07-28T10:13:01.737 INFO:teuthology.nuke:Not nuking smithi143.front.sepia.ceph.com because description doesn't match
2021-07-28T10:13:01.737 INFO:teuthology.nuke:Not nuking smithi110.front.sepia.ceph.com because description doesn't match