2021-04-21T23:42:01.932 INFO:root:teuthology version: 1.1.0-2713a3cd
2021-04-21T23:42:02.083 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi003.front.sepia.ceph.com' reimaging
2021-04-21T23:42:02.084 INFO:teuthology.lock.ops:Updating [ubuntu@smithi003.front.sepia.ceph.com]: reset os type and version on server
2021-04-21T23:42:02.084 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server
2021-04-21T23:42:02.100 INFO:teuthology.lock.ops:Node 'ubuntu@smithi003.front.sepia.ceph.com' reimaging is complete
2021-04-21T23:42:02.100 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging
2021-04-21T23:42:02.101 INFO:teuthology.lock.ops:Updating [ubuntu@smithi185.front.sepia.ceph.com]: reset os type and version on server
2021-04-21T23:42:02.101 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server
2021-04-21T23:42:02.116 INFO:teuthology.lock.ops:Node 'ubuntu@smithi185.front.sepia.ceph.com' reimaging is complete
2021-04-21T23:42:02.514 INFO:teuthology.provision.fog.smithi003:Scheduling deploy of rhel 8.3
2021-04-21T23:42:02.551 INFO:teuthology.provision.fog.smithi185:Scheduling deploy of rhel 8.3
2021-04-21T23:42:02.945 INFO:teuthology.orchestra.console:Power off smithi003
2021-04-21T23:42:03.064 INFO:teuthology.orchestra.console:Power off smithi185
2021-04-21T23:42:11.313 INFO:teuthology.orchestra.console:Power off for smithi003 completed
2021-04-21T23:42:11.414 INFO:teuthology.orchestra.console:Power on smithi003
2021-04-21T23:42:15.544 INFO:teuthology.orchestra.console:Power off for smithi185 completed
2021-04-21T23:42:15.645 INFO:teuthology.orchestra.console:Power on smithi185
2021-04-21T23:42:19.793 INFO:teuthology.orchestra.console:Power on for smithi003 completed
2021-04-21T23:42:19.894 INFO:teuthology.orchestra.console:Power on for smithi185 completed
2021-04-21T23:42:19.894 INFO:teuthology.provision.fog.smithi003:Waiting for deploy to finish
2021-04-21T23:42:19.995 INFO:teuthology.provision.fog.smithi185:Waiting for deploy to finish
2021-04-21T23:46:28.160 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-04-21T23:46:28.505 INFO:teuthology.provision.fog.smithi003:Node is ready
2021-04-21T23:46:28.520 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003.front.sepia.ceph.com
2021-04-21T23:46:28.576 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3 smithi003.front.sepia.ceph.com smithi003
2021-04-21T23:46:28.904 INFO:teuthology.provision.fog.smithi003:Deploy complete!
2021-04-21T23:47:07.552 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-04-21T23:47:07.840 INFO:teuthology.provision.fog.smithi185:Node is ready
2021-04-21T23:47:07.893 INFO:teuthology.orchestra.run.smithi185.stdout:smithi185.front.sepia.ceph.com
2021-04-21T23:47:07.949 INFO:teuthology.orchestra.run.smithi185.stdout:172.21.15.185 smithi185.front.sepia.ceph.com smithi185
2021-04-21T23:47:08.279 INFO:teuthology.provision.fog.smithi185:Deploy complete!
2021-04-21T23:47:08.397 INFO:teuthology.lock.ops:Checking smithi003.front.sepia.ceph.com
2021-04-21T23:47:08.397 INFO:teuthology.lock.ops:New key found. Updating...
2021-04-21T23:47:08.414 INFO:teuthology.lock.ops:Checking smithi185.front.sepia.ceph.com
2021-04-21T23:47:08.414 INFO:teuthology.lock.ops:New key found. Updating...
2021-04-21T23:47:08.429 INFO:teuthology.lock.ops:Updating [smithi003.front.sepia.ceph.com]: set os type and version on server
2021-04-21T23:47:08.431 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-04-21T23:47:08.722 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64
2021-04-21T23:47:08.776 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="Red Hat Enterprise Linux"
2021-04-21T23:47:08.777 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="8.3 (Ootpa)"
2021-04-21T23:47:08.777 INFO:teuthology.orchestra.run.smithi003.stdout:ID="rhel"
2021-04-21T23:47:08.777 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="fedora"
2021-04-21T23:47:08.777 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="8.3"
2021-04-21T23:47:08.777 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el8"
2021-04-21T23:47:08.777 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-04-21T23:47:08.777 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31"
2021-04-21T23:47:08.777 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-04-21T23:47:08.777 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://www.redhat.com/"
2021-04-21T23:47:08.777 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-04-21T23:47:08.777 INFO:teuthology.orchestra.run.smithi003.stdout:
2021-04-21T23:47:08.777 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-04-21T23:47:08.778 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-04-21T23:47:08.778 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-04-21T23:47:08.778 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-04-21T23:47:08.779 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server
2021-04-21T23:47:08.793 INFO:teuthology.lock.ops:Updating [smithi185.front.sepia.ceph.com]: set os type and version on server
2021-04-21T23:47:08.793 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-04-21T23:47:09.072 INFO:teuthology.orchestra.run.smithi185.stdout:x86_64
2021-04-21T23:47:09.125 INFO:teuthology.orchestra.run.smithi185.stdout:NAME="Red Hat Enterprise Linux"
2021-04-21T23:47:09.126 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION="8.3 (Ootpa)"
2021-04-21T23:47:09.126 INFO:teuthology.orchestra.run.smithi185.stdout:ID="rhel"
2021-04-21T23:47:09.126 INFO:teuthology.orchestra.run.smithi185.stdout:ID_LIKE="fedora"
2021-04-21T23:47:09.126 INFO:teuthology.orchestra.run.smithi185.stdout:VERSION_ID="8.3"
2021-04-21T23:47:09.126 INFO:teuthology.orchestra.run.smithi185.stdout:PLATFORM_ID="platform:el8"
2021-04-21T23:47:09.126 INFO:teuthology.orchestra.run.smithi185.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-04-21T23:47:09.126 INFO:teuthology.orchestra.run.smithi185.stdout:ANSI_COLOR="0;31"
2021-04-21T23:47:09.126 INFO:teuthology.orchestra.run.smithi185.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-04-21T23:47:09.126 INFO:teuthology.orchestra.run.smithi185.stdout:HOME_URL="https://www.redhat.com/"
2021-04-21T23:47:09.126 INFO:teuthology.orchestra.run.smithi185.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-04-21T23:47:09.127 INFO:teuthology.orchestra.run.smithi185.stdout:
2021-04-21T23:47:09.127 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-04-21T23:47:09.127 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-04-21T23:47:09.127 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-04-21T23:47:09.127 INFO:teuthology.orchestra.run.smithi185.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-04-21T23:47:09.127 INFO:teuthology.lock.ops:Updating smithi185.front.sepia.ceph.com on lock server
2021-04-21T23:47:09.211 INFO:teuthology.dispatcher.supervisor:Running job 6061963
2021-04-21T23:47:09.212 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2021-04-21_15:43:06-fs-wip-pdonnell-testing-20210421.004809-distro-basic-smithi/6061963 --name pdonnell-2021-04-21_15:43:06-fs-wip-pdonnell-testing-20210421.004809-distro-basic-smithi --description fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} -- /home/teuthworker/archive/pdonnell-2021-04-21_15:43:06-fs-wip-pdonnell-testing-20210421.004809-distro-basic-smithi/6061963/orig.config.yaml
2021-04-21T23:47:09.215 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2021-04-21_15:43:06-fs-wip-pdonnell-testing-20210421.004809-distro-basic-smithi/6061963
2021-04-21T23:47:09.215 INFO:teuthology.dispatcher.supervisor:Job PID: 22517
2021-04-21T23:47:09.215 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-04-22T11:48:06.228 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-04-22T11:48:06.553 INFO:teuthology.kill:Killing Pids: {22517}
2021-04-22T11:48:07.638 INFO:teuthology.kill:Nuking machines: ['smithi003', 'smithi185']
2021-04-22T11:48:08.608 INFO:teuthology.kill:2021-04-22 11:48:08,608.608 INFO:teuthology.nuke:targets:
2021-04-22T11:48:08.609 INFO:teuthology.kill:  smithi003.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsfpi0tBjJ/wZ41Qm8V/xMMmcE2YG6ODG5n3u76KsNRt2LEgVH00sLUSDBSVYcpj4DIIgzZdTLOtq+I0FjKfsx/kdN3ixeq2SXb0Pd+gIWBfONN08wYpw/twb0a/defJwXCu00E9NWLtzF+aiEChGjcryqnvBhrHT9AAzXK3kNt2ZgPKyehX9CZldDycuMbPvcmzrYoO/WseSWA6VXlrO/aRJfGqUDX3QhLmybasRp/8mn+dBQK//mbYMsPkOIeAZqR0CQajWuKrGLLA2K/zxR9jeih689BgZAoFWmaFqpt6KL+3X3vraegQ9gCnyGU8HWsgIykctHM2m48ARt7KHLBoWxZGGxqelaexNMLR7S4oFLdgYl4r0vTgU3SEUmoKTI9/rQqcHrl/kPf8RdCm4rOxcLdnTb2wVxJt0U5xM97gbvCrrv1Lo5qoJro9vIJuUNFpWlgzCcKnqjcjE8kME1PwGqI5dkrqbEI7+3xnjP9ui3wW7ew+0Ks4jURLO7y+c=
2021-04-22T11:48:08.609 INFO:teuthology.kill:  smithi185.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCx445/p9jwNRMKfOn31FuzVlXASy4fSXuwkWjMJXUj2R5WjT5NC2S8wuDjDsyMWxe6of63YmRg3h5guZJ8/IRIWlaKwnycidS9XiNWhknmnPRgUcpS+lW5ebtRt4NtvOY/cV3kCu1wmbQueIR9hLKUGpoURl9NUSjOR0O9fet7I9Zz5+URwOLS4Ve/jJ7zfREpwhI22dm2GnYEBa27lLVkFEVA4OEysL74/RmCif5CGwr4PH4Gd37xUznD6ijWPRf+Zbbk/wNb9TgZV0C8CfrkzNr9K/X/GxChcubZkobMz1f1d6pVElNDaOARNwvlUWbip51HTr4v/7givguAfyrvHDUb2Wm60//CnUNugRMCDE0voS38HvFH5rAne/MRtybGrFNwM6Wzp72pZcLiJHiVRYhsCFKm8WqkuD4Z+VKNZbMtBI+fnx8tYW2nsIf/mpWjKttdXpf/HURp/4IcJ06KzOWUuZ9FMUbSio5H/LTZ5HKb49fZS81LfKIVgzgibNk=
2021-04-22T11:48:08.609 INFO:teuthology.kill:2021-04-22 11:48:08,608.608 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-22T11:48:08.611 INFO:teuthology.kill:2021-04-22 11:48:08,611.611 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-22T11:48:20.030 INFO:teuthology.kill:2021-04-22 11:48:20,029.029 INFO:teuthology.orchestra.console:Power off smithi003
2021-04-22T11:48:20.045 INFO:teuthology.kill:2021-04-22 11:48:20,045.045 INFO:teuthology.orchestra.console:Power off smithi185
2021-04-22T11:48:32.538 INFO:teuthology.kill:2021-04-22 11:48:32,538.538 INFO:teuthology.orchestra.console:Power off for smithi185 completed
2021-04-22T11:48:32.543 INFO:teuthology.kill:2021-04-22 11:48:32,543.543 INFO:teuthology.orchestra.console:Power off for smithi003 completed
2021-04-22T11:48:32.786 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0']
2021-04-22T11:48:32.787 INFO:teuthology.task.internal:roles: ubuntu@smithi185.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']
2021-04-22T11:48:32.787 INFO:teuthology.misc:Compressing logs...
2021-04-22T11:48:32.787 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-04-22T11:48:51.163 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/remote.py", line 215, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-04-22T11:48:51.236 INFO:teuthology.kill:No teuthology processes running
2021-04-22T11:49:13.692 INFO:teuthology.kill:Nuking machines: ['smithi003', 'smithi185']
2021-04-22T11:49:15.353 INFO:teuthology.kill:2021-04-22 11:49:15,353.353 INFO:teuthology.nuke:targets:
2021-04-22T11:49:15.354 INFO:teuthology.kill:  smithi003.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsfpi0tBjJ/wZ41Qm8V/xMMmcE2YG6ODG5n3u76KsNRt2LEgVH00sLUSDBSVYcpj4DIIgzZdTLOtq+I0FjKfsx/kdN3ixeq2SXb0Pd+gIWBfONN08wYpw/twb0a/defJwXCu00E9NWLtzF+aiEChGjcryqnvBhrHT9AAzXK3kNt2ZgPKyehX9CZldDycuMbPvcmzrYoO/WseSWA6VXlrO/aRJfGqUDX3QhLmybasRp/8mn+dBQK//mbYMsPkOIeAZqR0CQajWuKrGLLA2K/zxR9jeih689BgZAoFWmaFqpt6KL+3X3vraegQ9gCnyGU8HWsgIykctHM2m48ARt7KHLBoWxZGGxqelaexNMLR7S4oFLdgYl4r0vTgU3SEUmoKTI9/rQqcHrl/kPf8RdCm4rOxcLdnTb2wVxJt0U5xM97gbvCrrv1Lo5qoJro9vIJuUNFpWlgzCcKnqjcjE8kME1PwGqI5dkrqbEI7+3xnjP9ui3wW7ew+0Ks4jURLO7y+c=
2021-04-22T11:49:15.354 INFO:teuthology.kill:  smithi185.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCx445/p9jwNRMKfOn31FuzVlXASy4fSXuwkWjMJXUj2R5WjT5NC2S8wuDjDsyMWxe6of63YmRg3h5guZJ8/IRIWlaKwnycidS9XiNWhknmnPRgUcpS+lW5ebtRt4NtvOY/cV3kCu1wmbQueIR9hLKUGpoURl9NUSjOR0O9fet7I9Zz5+URwOLS4Ve/jJ7zfREpwhI22dm2GnYEBa27lLVkFEVA4OEysL74/RmCif5CGwr4PH4Gd37xUznD6ijWPRf+Zbbk/wNb9TgZV0C8CfrkzNr9K/X/GxChcubZkobMz1f1d6pVElNDaOARNwvlUWbip51HTr4v/7givguAfyrvHDUb2Wm60//CnUNugRMCDE0voS38HvFH5rAne/MRtybGrFNwM6Wzp72pZcLiJHiVRYhsCFKm8WqkuD4Z+VKNZbMtBI+fnx8tYW2nsIf/mpWjKttdXpf/HURp/4IcJ06KzOWUuZ9FMUbSio5H/LTZ5HKb49fZS81LfKIVgzgibNk=
2021-04-22T11:49:15.378 INFO:teuthology.kill:2021-04-22 11:49:15,377.377 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-22T11:49:15.390 INFO:teuthology.kill:2021-04-22 11:49:15,390.390 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-22T11:49:15.491 INFO:teuthology.kill:2021-04-22 11:49:15,491.491 INFO:teuthology.orchestra.console:Power off smithi003
2021-04-22T11:49:15.508 INFO:teuthology.kill:2021-04-22 11:49:15,508.508 INFO:teuthology.orchestra.console:Power off smithi185
2021-04-22T11:49:15.663 INFO:teuthology.kill:2021-04-22 11:49:15,662.662 INFO:teuthology.orchestra.console:Power off for smithi185 completed
2021-04-22T11:49:15.670 INFO:teuthology.kill:2021-04-22 11:49:15,670.670 INFO:teuthology.orchestra.console:Power off for smithi003 completed
2021-04-22T11:49:15.823 INFO:teuthology.kill:2021-04-22 11:49:15,823.823 INFO:teuthology.lock.ops:unlocked smithi185.front.sepia.ceph.com
2021-04-22T11:49:15.866 INFO:teuthology.kill:2021-04-22 11:49:15,866.866 INFO:teuthology.lock.ops:unlocked smithi003.front.sepia.ceph.com
2021-04-22T11:51:16.242 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-04-22T11:51:16.390 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-04-22T11:51:16.390 INFO:teuthology.nuke:Checking targets against current locks
2021-04-22T11:51:16.482 INFO:teuthology.nuke:Not nuking smithi185.front.sepia.ceph.com because description doesn't match
2021-04-22T11:51:16.482 INFO:teuthology.nuke:Not nuking smithi003.front.sepia.ceph.com because description doesn't match