2021-03-13T09:13:52.688 INFO:root:teuthology version: 1.1.0-b9656917 2021-03-13T09:13:52.786 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi025.front.sepia.ceph.com' reimaging 2021-03-13T09:13:52.787 INFO:teuthology.lock.ops:Updating [ubuntu@smithi025.front.sepia.ceph.com]: reset os type and version on server 2021-03-13T09:13:52.787 INFO:teuthology.lock.ops:Updating smithi025.front.sepia.ceph.com on lock server 2021-03-13T09:13:52.801 INFO:teuthology.lock.ops:Node 'ubuntu@smithi025.front.sepia.ceph.com' reimaging is complete 2021-03-13T09:13:52.802 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi094.front.sepia.ceph.com' reimaging 2021-03-13T09:13:52.802 INFO:teuthology.lock.ops:Updating [ubuntu@smithi094.front.sepia.ceph.com]: reset os type and version on server 2021-03-13T09:13:52.803 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2021-03-13T09:13:52.819 INFO:teuthology.lock.ops:Node 'ubuntu@smithi094.front.sepia.ceph.com' reimaging is complete 2021-03-13T09:13:53.014 INFO:teuthology.provision.fog.smithi025:Scheduling deploy of rhel 8.3 2021-03-13T09:13:53.027 INFO:teuthology.provision.fog.smithi094:Scheduling deploy of rhel 8.3 2021-03-13T09:13:53.316 INFO:teuthology.orchestra.console:Power off smithi025 2021-03-13T09:13:53.340 INFO:teuthology.orchestra.console:Power off smithi094 2021-03-13T09:14:01.700 INFO:teuthology.orchestra.console:Power off for smithi025 completed 2021-03-13T09:14:01.733 INFO:teuthology.orchestra.console:Power off for smithi094 completed 2021-03-13T09:14:01.804 INFO:teuthology.orchestra.console:Power on smithi025 2021-03-13T09:14:01.840 INFO:teuthology.orchestra.console:Power on smithi094 2021-03-13T09:14:10.183 INFO:teuthology.orchestra.console:Power on for smithi025 completed 2021-03-13T09:14:10.201 INFO:teuthology.orchestra.console:Power on for smithi094 completed 2021-03-13T09:14:10.285 INFO:teuthology.provision.fog.smithi025:Waiting for deploy to finish 2021-03-13T09:14:10.303 INFO:teuthology.provision.fog.smithi094:Waiting for deploy to finish 2021-03-13T09:18:42.240 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-13T09:18:42.552 INFO:teuthology.provision.fog.smithi094:Node is ready 2021-03-13T09:18:42.567 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094.front.sepia.ceph.com 2021-03-13T09:18:42.621 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94 smithi094.front.sepia.ceph.com smithi094 2021-03-13T09:18:42.934 INFO:teuthology.provision.fog.smithi094:Deploy complete! 2021-03-13T09:19:39.610 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-13T09:19:39.923 INFO:teuthology.provision.fog.smithi025:Node is ready 2021-03-13T09:19:39.939 INFO:teuthology.orchestra.run.smithi025.stdout:smithi025.front.sepia.ceph.com 2021-03-13T09:19:39.994 INFO:teuthology.orchestra.run.smithi025.stdout:172.21.15.25 smithi025.front.sepia.ceph.com smithi025 2021-03-13T09:19:40.327 INFO:teuthology.provision.fog.smithi025:Deploy complete! 2021-03-13T09:19:40.455 INFO:teuthology.lock.ops:Checking smithi025.front.sepia.ceph.com 2021-03-13T09:19:40.456 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-13T09:19:40.473 INFO:teuthology.lock.ops:Checking smithi094.front.sepia.ceph.com 2021-03-13T09:19:40.473 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-13T09:19:40.491 INFO:teuthology.lock.ops:Updating [smithi025.front.sepia.ceph.com]: set os type and version on server 2021-03-13T09:19:40.491 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-13T09:19:40.775 INFO:teuthology.orchestra.run.smithi025.stdout:x86_64 2021-03-13T09:19:40.829 INFO:teuthology.orchestra.run.smithi025.stdout:NAME="Red Hat Enterprise Linux" 2021-03-13T09:19:40.830 INFO:teuthology.orchestra.run.smithi025.stdout:VERSION="8.3 (Ootpa)" 2021-03-13T09:19:40.830 INFO:teuthology.orchestra.run.smithi025.stdout:ID="rhel" 2021-03-13T09:19:40.830 INFO:teuthology.orchestra.run.smithi025.stdout:ID_LIKE="fedora" 2021-03-13T09:19:40.831 INFO:teuthology.orchestra.run.smithi025.stdout:VERSION_ID="8.3" 2021-03-13T09:19:40.831 INFO:teuthology.orchestra.run.smithi025.stdout:PLATFORM_ID="platform:el8" 2021-03-13T09:19:40.831 INFO:teuthology.orchestra.run.smithi025.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-03-13T09:19:40.831 INFO:teuthology.orchestra.run.smithi025.stdout:ANSI_COLOR="0;31" 2021-03-13T09:19:40.832 INFO:teuthology.orchestra.run.smithi025.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-03-13T09:19:40.832 INFO:teuthology.orchestra.run.smithi025.stdout:HOME_URL="https://www.redhat.com/" 2021-03-13T09:19:40.832 INFO:teuthology.orchestra.run.smithi025.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-03-13T09:19:40.832 INFO:teuthology.orchestra.run.smithi025.stdout: 2021-03-13T09:19:40.832 INFO:teuthology.orchestra.run.smithi025.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-03-13T09:19:40.833 INFO:teuthology.orchestra.run.smithi025.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-03-13T09:19:40.833 INFO:teuthology.orchestra.run.smithi025.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-03-13T09:19:40.833 INFO:teuthology.orchestra.run.smithi025.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-03-13T09:19:40.834 INFO:teuthology.lock.ops:Updating smithi025.front.sepia.ceph.com on lock server 2021-03-13T09:19:40.852 INFO:teuthology.lock.ops:Updating [smithi094.front.sepia.ceph.com]: set os type and version on server 2021-03-13T09:19:40.853 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-13T09:19:41.143 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2021-03-13T09:19:41.197 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="Red Hat Enterprise Linux" 2021-03-13T09:19:41.198 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="8.3 (Ootpa)" 2021-03-13T09:19:41.198 INFO:teuthology.orchestra.run.smithi094.stdout:ID="rhel" 2021-03-13T09:19:41.198 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="fedora" 2021-03-13T09:19:41.198 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="8.3" 2021-03-13T09:19:41.198 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el8" 2021-03-13T09:19:41.199 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-03-13T09:19:41.199 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2021-03-13T09:19:41.199 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-03-13T09:19:41.199 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://www.redhat.com/" 2021-03-13T09:19:41.200 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-03-13T09:19:41.200 INFO:teuthology.orchestra.run.smithi094.stdout: 2021-03-13T09:19:41.200 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-03-13T09:19:41.200 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-03-13T09:19:41.200 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-03-13T09:19:41.201 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-03-13T09:19:41.201 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2021-03-13T09:19:41.279 INFO:teuthology.dispatcher.supervisor:Running job 5961051 2021-03-13T09:19:41.279 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/bin/teuthology -v --owner scheduled_kchai@teuthology --archive /home/teuthworker/archive/kchai-2021-03-13_08:16:32-rados-wip-kefu-testing-2021-03-13-1449-distro-basic-smithi/5961051 --name kchai-2021-03-13_08:16:32-rados-wip-kefu-testing-2021-03-13-1449-distro-basic-smithi --description rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} -- /home/teuthworker/archive/kchai-2021-03-13_08:16:32-rados-wip-kefu-testing-2021-03-13-1449-distro-basic-smithi/5961051/orig.config.yaml 2021-03-13T09:19:41.283 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/kchai-2021-03-13_08:16:32-rados-wip-kefu-testing-2021-03-13-1449-distro-basic-smithi/5961051 2021-03-13T09:19:41.284 INFO:teuthology.dispatcher.supervisor:Job PID: 28542 2021-03-13T09:19:41.284 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-03-13T21:20:33.019 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-03-13T21:20:33.322 INFO:teuthology.kill:Killing Pids: {28542} 2021-03-13T21:20:35.363 INFO:teuthology.kill:Nuking machines: ['smithi025', 'smithi094'] 2021-03-13T21:20:37.755 INFO:teuthology.kill:2021-03-13 21:20:37,754.754 INFO:teuthology.nuke:targets: 2021-03-13T21:20:37.755 INFO:teuthology.kill: smithi025.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBBpoiN2xOub5CmEdXhV1XaWNlF7rZAgTvnYFPCL8vaPd2J+AZ+KFrClQIDU51UfvnPYyzYRmC5i7HzvNBreMzJyzpfcd46hNxbhrjBuwrbF+VCbYgh1iJx3sXSntSygSOFKY0puWB+RV4/HnVOn7XaPLJFNGEGINGaA93uZJmCPT3pZccYxJuTZ8LsIe72UZfCxa2MtIGp3AZeHoND7kqJ1aDuyAAcKxcUgvzyB3UkOXP7oAPcYg6XJAtHGMFl/ZRaLaXvD4+h6QvwtpHzWc1QJvfAuMsx/WkcBJ752anP8p4rw0213PlkPjEAGCzNRB0vOB25WAdUu7lKPZ4wTayF1p4WQRLsI13V1Rpb/F/uOxxt98Q5uuyZgSiUc6gsXeSbsVVu5U/ptmJa7wezgrV0tOxe2cUIKyXxYQ0GdbnT00oxl+ztDCaLdqu3Gk0aT0SAaG2gbYZ9BtTm+CJy6LgtxMo2qE+QVfZ15mwGLy3tcMyHKhNfcp7t9jdhYuDB9E= 2021-03-13T21:20:37.756 INFO:teuthology.kill: smithi094.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMdnJxdX0MMbh5roZ7gJ7iYEkXCmo9350h/HYwodV5SY1zhOvzMWPyL2lKikPfZnZDKV54Y6946KenRbGVouBj1Kmmq9A7RB2CaVvltompqhqVkk465+axaujEpm18M/j+AF+WfMHvZHFGLUHq6qlwbOFIHtQIXsU0sXm/7oxNzCJVrjMS0wg5f8lhxFi4rSD70+1OJCUaU3iDwKlMwKq5FfaAt8FOHeoX68+RKP7EKChEpmrOxDEoloqq9UEMLhk+IBiPHxRnxLWxVOuXo8BCmdES2MUX0Q7FgdWK5h+8PbsJM5vdOY8buqTNsD5kPhb6RbeIUJH4L7+rUldcoNmflSGl11QgubyIOpJ6Dp3cGYxIwVDOHdwpfhMP1zBlaXaWPV5mDRwo0skI0ju7ZLvKe6FvWZyORqfj5DibKsobA5rhQx12ZG0NktHDkXpOVSx6jUbgbxy82nBldfJXfwW1zsJb1n07FgslSs9owvhQodJ7rWh/Ma69KB2quT3+nrc= 2021-03-13T21:20:37.756 INFO:teuthology.kill:2021-03-13 21:20:37,755.755 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-13T21:20:37.774 INFO:teuthology.kill:2021-03-13 21:20:37,757.757 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-13T21:20:37.888 INFO:teuthology.kill:2021-03-13 21:20:37,887.887 INFO:teuthology.orchestra.console:Power off smithi094 2021-03-13T21:20:37.953 INFO:teuthology.kill:2021-03-13 21:20:37,952.952 INFO:teuthology.orchestra.console:Power off smithi025 2021-03-13T21:20:50.554 INFO:teuthology.kill:2021-03-13 21:20:50,554.554 INFO:teuthology.orchestra.console:Power off for smithi094 completed 2021-03-13T21:20:50.575 INFO:teuthology.kill:2021-03-13 21:20:50,575.575 INFO:teuthology.orchestra.console:Power off for smithi025 completed 2021-03-13T21:20:51.003 INFO:teuthology.task.internal:roles: ubuntu@smithi025.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-03-13T21:20:51.004 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-03-13T21:20:51.004 INFO:teuthology.misc:Compressing logs... 2021-03-13T21:20:51.005 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-13T21:21:51.079 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/misc.py", line 1374, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-03-13T21:21:51.194 INFO:teuthology.kill:No teuthology processes running 2021-03-13T21:21:53.163 INFO:teuthology.kill:Nuking machines: ['smithi025', 'smithi094'] 2021-03-13T21:21:55.051 INFO:teuthology.kill:2021-03-13 21:21:55,051.051 INFO:teuthology.nuke:targets: 2021-03-13T21:21:55.052 INFO:teuthology.kill: smithi025.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBBpoiN2xOub5CmEdXhV1XaWNlF7rZAgTvnYFPCL8vaPd2J+AZ+KFrClQIDU51UfvnPYyzYRmC5i7HzvNBreMzJyzpfcd46hNxbhrjBuwrbF+VCbYgh1iJx3sXSntSygSOFKY0puWB+RV4/HnVOn7XaPLJFNGEGINGaA93uZJmCPT3pZccYxJuTZ8LsIe72UZfCxa2MtIGp3AZeHoND7kqJ1aDuyAAcKxcUgvzyB3UkOXP7oAPcYg6XJAtHGMFl/ZRaLaXvD4+h6QvwtpHzWc1QJvfAuMsx/WkcBJ752anP8p4rw0213PlkPjEAGCzNRB0vOB25WAdUu7lKPZ4wTayF1p4WQRLsI13V1Rpb/F/uOxxt98Q5uuyZgSiUc6gsXeSbsVVu5U/ptmJa7wezgrV0tOxe2cUIKyXxYQ0GdbnT00oxl+ztDCaLdqu3Gk0aT0SAaG2gbYZ9BtTm+CJy6LgtxMo2qE+QVfZ15mwGLy3tcMyHKhNfcp7t9jdhYuDB9E= 2021-03-13T21:21:55.052 INFO:teuthology.kill: smithi094.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMdnJxdX0MMbh5roZ7gJ7iYEkXCmo9350h/HYwodV5SY1zhOvzMWPyL2lKikPfZnZDKV54Y6946KenRbGVouBj1Kmmq9A7RB2CaVvltompqhqVkk465+axaujEpm18M/j+AF+WfMHvZHFGLUHq6qlwbOFIHtQIXsU0sXm/7oxNzCJVrjMS0wg5f8lhxFi4rSD70+1OJCUaU3iDwKlMwKq5FfaAt8FOHeoX68+RKP7EKChEpmrOxDEoloqq9UEMLhk+IBiPHxRnxLWxVOuXo8BCmdES2MUX0Q7FgdWK5h+8PbsJM5vdOY8buqTNsD5kPhb6RbeIUJH4L7+rUldcoNmflSGl11QgubyIOpJ6Dp3cGYxIwVDOHdwpfhMP1zBlaXaWPV5mDRwo0skI0ju7ZLvKe6FvWZyORqfj5DibKsobA5rhQx12ZG0NktHDkXpOVSx6jUbgbxy82nBldfJXfwW1zsJb1n07FgslSs9owvhQodJ7rWh/Ma69KB2quT3+nrc= 2021-03-13T21:21:55.075 INFO:teuthology.kill:2021-03-13 21:21:55,074.074 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-13T21:21:55.077 INFO:teuthology.kill:2021-03-13 21:21:55,077.077 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-13T21:21:55.176 INFO:teuthology.kill:2021-03-13 21:21:55,174.174 INFO:teuthology.orchestra.console:Power off smithi025 2021-03-13T21:21:55.257 INFO:teuthology.kill:2021-03-13 21:21:55,257.257 INFO:teuthology.orchestra.console:Power off smithi094 2021-03-13T21:21:55.453 INFO:teuthology.kill:2021-03-13 21:21:55,453.453 INFO:teuthology.orchestra.console:Power off for smithi025 completed 2021-03-13T21:21:55.483 INFO:teuthology.kill:2021-03-13 21:21:55,483.483 INFO:teuthology.orchestra.console:Power off for smithi094 completed 2021-03-13T21:21:55.601 INFO:teuthology.kill:2021-03-13 21:21:55,600.600 INFO:teuthology.lock.ops:unlocked smithi025.front.sepia.ceph.com 2021-03-13T21:21:55.628 INFO:teuthology.kill:2021-03-13 21:21:55,628.628 INFO:teuthology.lock.ops:unlocked smithi094.front.sepia.ceph.com 2021-03-13T21:23:56.143 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-03-13T21:23:56.288 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-03-13T21:23:56.288 INFO:teuthology.nuke:Checking targets against current locks 2021-03-13T21:23:56.388 INFO:teuthology.nuke:Not nuking smithi094.front.sepia.ceph.com because description doesn't match 2021-03-13T21:23:56.388 INFO:teuthology.nuke:Not nuking smithi025.front.sepia.ceph.com because description doesn't match