2021-09-06T12:20:23.427 INFO:root:teuthology version: 1.1.0-df69d6db 2021-09-06T12:20:23.526 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi041.front.sepia.ceph.com' reimaging 2021-09-06T12:20:23.527 INFO:teuthology.lock.ops:Updating [ubuntu@smithi041.front.sepia.ceph.com]: reset os type and version on server 2021-09-06T12:20:23.527 INFO:teuthology.lock.ops:Updating smithi041.front.sepia.ceph.com on lock server 2021-09-06T12:20:23.544 INFO:teuthology.lock.ops:Node 'ubuntu@smithi041.front.sepia.ceph.com' reimaging is complete 2021-09-06T12:20:23.544 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi052.front.sepia.ceph.com' reimaging 2021-09-06T12:20:23.545 INFO:teuthology.lock.ops:Updating [ubuntu@smithi052.front.sepia.ceph.com]: reset os type and version on server 2021-09-06T12:20:23.545 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2021-09-06T12:20:23.563 INFO:teuthology.lock.ops:Node 'ubuntu@smithi052.front.sepia.ceph.com' reimaging is complete 2021-09-06T12:20:23.750 INFO:teuthology.provision.fog.smithi041:Scheduling deploy of rhel 8.4 2021-09-06T12:20:23.765 INFO:teuthology.provision.fog.smithi052:Scheduling deploy of rhel 8.4 2021-09-06T12:20:24.051 INFO:teuthology.orchestra.console:Power off smithi041 2021-09-06T12:20:24.080 INFO:teuthology.orchestra.console:Power off smithi052 2021-09-06T12:20:24.215 INFO:teuthology.orchestra.console:Power off for smithi041 completed 2021-09-06T12:20:24.227 INFO:teuthology.orchestra.console:Power off for smithi052 completed 2021-09-06T12:20:24.317 INFO:teuthology.orchestra.console:Power on smithi041 2021-09-06T12:20:24.329 INFO:teuthology.orchestra.console:Power on smithi052 2021-09-06T12:20:32.687 INFO:teuthology.orchestra.console:Power on for smithi041 completed 2021-09-06T12:20:32.692 INFO:teuthology.orchestra.console:Power on for smithi052 completed 2021-09-06T12:20:32.789 INFO:teuthology.provision.fog.smithi041:Waiting for deploy to finish 2021-09-06T12:20:32.794 INFO:teuthology.provision.fog.smithi052:Waiting for deploy to finish 2021-09-06T12:24:37.736 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-09-06T12:24:37.973 INFO:teuthology.provision.fog.smithi052:Node is ready 2021-09-06T12:24:37.989 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052.front.sepia.ceph.com 2021-09-06T12:24:38.045 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52 smithi052.front.sepia.ceph.com smithi052 2021-09-06T12:24:38.376 INFO:teuthology.provision.fog.smithi052:Deploy complete! 2021-09-06T12:24:46.685 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-09-06T12:24:46.904 INFO:teuthology.provision.fog.smithi041:Node is ready 2021-09-06T12:24:46.920 INFO:teuthology.orchestra.run.smithi041.stdout:smithi041.front.sepia.ceph.com 2021-09-06T12:24:46.976 INFO:teuthology.orchestra.run.smithi041.stdout:172.21.15.41 smithi041.front.sepia.ceph.com smithi041 2021-09-06T12:24:47.315 INFO:teuthology.provision.fog.smithi041:Deploy complete! 2021-09-06T12:24:47.447 INFO:teuthology.lock.ops:Checking smithi041.front.sepia.ceph.com 2021-09-06T12:24:47.447 INFO:teuthology.lock.ops:New key found. Updating... 2021-09-06T12:24:47.466 INFO:teuthology.lock.ops:Checking smithi052.front.sepia.ceph.com 2021-09-06T12:24:47.467 INFO:teuthology.lock.ops:New key found. Updating... 2021-09-06T12:24:47.482 INFO:teuthology.lock.ops:Updating [smithi041.front.sepia.ceph.com]: set os type and version on server 2021-09-06T12:24:47.483 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-09-06T12:24:47.707 INFO:teuthology.orchestra.run.smithi041.stdout:x86_64 2021-09-06T12:24:47.761 INFO:teuthology.orchestra.run.smithi041.stdout:NAME="Red Hat Enterprise Linux" 2021-09-06T12:24:47.761 INFO:teuthology.orchestra.run.smithi041.stdout:VERSION="8.4 (Ootpa)" 2021-09-06T12:24:47.762 INFO:teuthology.orchestra.run.smithi041.stdout:ID="rhel" 2021-09-06T12:24:47.762 INFO:teuthology.orchestra.run.smithi041.stdout:ID_LIKE="fedora" 2021-09-06T12:24:47.762 INFO:teuthology.orchestra.run.smithi041.stdout:VERSION_ID="8.4" 2021-09-06T12:24:47.762 INFO:teuthology.orchestra.run.smithi041.stdout:PLATFORM_ID="platform:el8" 2021-09-06T12:24:47.763 INFO:teuthology.orchestra.run.smithi041.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-09-06T12:24:47.763 INFO:teuthology.orchestra.run.smithi041.stdout:ANSI_COLOR="0;31" 2021-09-06T12:24:47.763 INFO:teuthology.orchestra.run.smithi041.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-09-06T12:24:47.763 INFO:teuthology.orchestra.run.smithi041.stdout:HOME_URL="https://www.redhat.com/" 2021-09-06T12:24:47.764 INFO:teuthology.orchestra.run.smithi041.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-09-06T12:24:47.764 INFO:teuthology.orchestra.run.smithi041.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-09-06T12:24:47.764 INFO:teuthology.orchestra.run.smithi041.stdout: 2021-09-06T12:24:47.764 INFO:teuthology.orchestra.run.smithi041.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-09-06T12:24:47.764 INFO:teuthology.orchestra.run.smithi041.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-09-06T12:24:47.765 INFO:teuthology.orchestra.run.smithi041.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-09-06T12:24:47.765 INFO:teuthology.orchestra.run.smithi041.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-09-06T12:24:47.766 INFO:teuthology.lock.ops:Updating smithi041.front.sepia.ceph.com on lock server 2021-09-06T12:24:47.782 INFO:teuthology.lock.ops:Updating [smithi052.front.sepia.ceph.com]: set os type and version on server 2021-09-06T12:24:47.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-09-06T12:24:48.021 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64 2021-09-06T12:24:48.077 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="Red Hat Enterprise Linux" 2021-09-06T12:24:48.078 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="8.4 (Ootpa)" 2021-09-06T12:24:48.078 INFO:teuthology.orchestra.run.smithi052.stdout:ID="rhel" 2021-09-06T12:24:48.078 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="fedora" 2021-09-06T12:24:48.078 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="8.4" 2021-09-06T12:24:48.079 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el8" 2021-09-06T12:24:48.079 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-09-06T12:24:48.079 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31" 2021-09-06T12:24:48.079 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-09-06T12:24:48.079 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://www.redhat.com/" 2021-09-06T12:24:48.080 INFO:teuthology.orchestra.run.smithi052.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-09-06T12:24:48.080 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-09-06T12:24:48.080 INFO:teuthology.orchestra.run.smithi052.stdout: 2021-09-06T12:24:48.080 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-09-06T12:24:48.081 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-09-06T12:24:48.081 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-09-06T12:24:48.081 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-09-06T12:24:48.081 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2021-09-06T12:24:48.149 INFO:teuthology.dispatcher.supervisor:Running job 6376857 2021-09-06T12:24:48.149 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/virtualenv/bin/teuthology -v --owner scheduled_soumyakoduri@teuthology --archive /home/teuthworker/archive/soumyakoduri-2021-09-06_10:30:05-rgw-wip-skoduri-lua-distro-basic-smithi/6376857 --name soumyakoduri-2021-09-06_10:30:05-rgw-wip-skoduri-lua-distro-basic-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{rhel_8} tasks/test_multi} -- /home/teuthworker/archive/soumyakoduri-2021-09-06_10:30:05-rgw-wip-skoduri-lua-distro-basic-smithi/6376857/orig.config.yaml 2021-09-06T12:24:48.153 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/soumyakoduri-2021-09-06_10:30:05-rgw-wip-skoduri-lua-distro-basic-smithi/6376857 2021-09-06T12:24:48.153 INFO:teuthology.dispatcher.supervisor:Job PID: 20386 2021-09-06T12:24:48.154 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-09-07T00:25:31.040 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-09-07T00:25:31.192 INFO:teuthology.kill:Killing Pids: {20386} 2021-09-07T00:25:32.179 INFO:teuthology.kill:Nuking machines: ['smithi041', 'smithi052'] 2021-09-07T00:25:33.043 INFO:teuthology.kill:2021-09-07 00:25:33,042.042 INFO:teuthology.nuke:targets: 2021-09-07T00:25:33.044 INFO:teuthology.kill: smithi041.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtKEAsKjpnirpPJZlrVlDB6EUcJevmYskuRXgQf6uMRHU+EU2BE/X82jOB1CqMpIfr8BU2Ji7R/5NNG5JscS8FgZ06sUAoAo8g06h6Qf4JVI/JcRlmNOALGPSbdOQVranDh1SQn+Jhs02F/M+I1xdxwd63mXbfaGGEe4YpIuYozFVISgrXDNjEm3BOxUP9BUhWNyzVIFjB4uLvNKqP03Zpp1amOEJ38pXu9IlCC1RMFhSqw7eWQDih957x+Pl8LbUBtQlq4b+XRrkSCL+OwdboW3keev60KLitsQqYpvGlAdynHhRrC0tWiMOmTQa4fEyzf6fZFZVAC9AYG1X7TIiQg/cVokcFCG2iymZBE0LXbzGRulXWiDE4orhmlt818fI+SCJoJ7exNpUpe/7H7bdLDncrY7KVO4fbaK8IGp32jmHMzYdMJ6yRw9Io25LtFFY/Obz0EZW4UpV+ccEpwtuFKR0qjvAQjUdurnqHU7dB5Ujdi4o4Xv/sgr2AO5KEChc= 2021-09-07T00:25:33.044 INFO:teuthology.kill: smithi052.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDT1FWubehwPPALY5EhmmXs2T9Kf7zlSb7OkqXs4OzGfyd3xKEh38q10xEE4m6rgXG+GBpWHBJQvZUJSyBukInqY5N7EJ0/nzsyPQt/c7DcWSgvFuE4sA/ihpjg5ZikISU523gDbtc9AXqYX6wUouhh3LJ1ZI25LwHZQCkFON34Sii0I8wn58/9RtZY8AZ8C86UhbYZbVQvYwQJOYVPprf0rXSiQy4A1sZR4J/jm4oVU6VBtkzvzOBZNME011hmU5yWJ/RB6CPaf5nCO82HEBJorqfQ/QnJZeq3m5Jf87+mnOtitKe/ml8M6a0fcBq6XnGvPoD4o9bzRtrNg478pVxFOU+dbOI/YaV7zAtIWER2V5LRZgDzdmoFXBBxABf3h9PSiX3yPvT3PGbeNHAQctnGqLo1fYhBBKMt2mfxFkq/J5K+7GY8Z/Z1IlxE1qB479mOS+AiNxV6VwljlP5WIA2EkKjRAEvNjwr+xh12V2jzSNzH/IAYZ8R4/VfkmE6MUR8= 2021-09-07T00:25:33.044 INFO:teuthology.kill:2021-09-07 00:25:33,043.043 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-07T00:25:33.045 INFO:teuthology.kill:2021-09-07 00:25:33,045.045 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-07T00:25:33.104 INFO:teuthology.kill:2021-09-07 00:25:33,103.103 INFO:teuthology.orchestra.console:Power off smithi052 2021-09-07T00:25:33.116 INFO:teuthology.kill:2021-09-07 00:25:33,116.116 INFO:teuthology.orchestra.console:Power off smithi041 2021-09-07T00:25:41.476 INFO:teuthology.kill:2021-09-07 00:25:41,475.475 INFO:teuthology.orchestra.console:Power off for smithi052 completed 2021-09-07T00:25:45.592 INFO:teuthology.kill:2021-09-07 00:25:45,591.591 INFO:teuthology.orchestra.console:Power off for smithi041 completed 2021-09-07T00:25:45.804 INFO:teuthology.task.internal:roles: ubuntu@smithi041.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2021-09-07T00:25:45.805 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2021-09-07T00:25:45.805 INFO:teuthology.misc:Compressing logs... 2021-09-07T00:25:45.806 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-09-07T00:26:45.870 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-09-07T00:26:45.915 INFO:teuthology.kill:No teuthology processes running 2021-09-07T00:26:46.928 INFO:teuthology.kill:Nuking machines: ['smithi041', 'smithi052'] 2021-09-07T00:26:47.778 INFO:teuthology.kill:2021-09-07 00:26:47,777.777 INFO:teuthology.nuke:targets: 2021-09-07T00:26:47.779 INFO:teuthology.kill: smithi041.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtKEAsKjpnirpPJZlrVlDB6EUcJevmYskuRXgQf6uMRHU+EU2BE/X82jOB1CqMpIfr8BU2Ji7R/5NNG5JscS8FgZ06sUAoAo8g06h6Qf4JVI/JcRlmNOALGPSbdOQVranDh1SQn+Jhs02F/M+I1xdxwd63mXbfaGGEe4YpIuYozFVISgrXDNjEm3BOxUP9BUhWNyzVIFjB4uLvNKqP03Zpp1amOEJ38pXu9IlCC1RMFhSqw7eWQDih957x+Pl8LbUBtQlq4b+XRrkSCL+OwdboW3keev60KLitsQqYpvGlAdynHhRrC0tWiMOmTQa4fEyzf6fZFZVAC9AYG1X7TIiQg/cVokcFCG2iymZBE0LXbzGRulXWiDE4orhmlt818fI+SCJoJ7exNpUpe/7H7bdLDncrY7KVO4fbaK8IGp32jmHMzYdMJ6yRw9Io25LtFFY/Obz0EZW4UpV+ccEpwtuFKR0qjvAQjUdurnqHU7dB5Ujdi4o4Xv/sgr2AO5KEChc= 2021-09-07T00:26:47.779 INFO:teuthology.kill: smithi052.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDT1FWubehwPPALY5EhmmXs2T9Kf7zlSb7OkqXs4OzGfyd3xKEh38q10xEE4m6rgXG+GBpWHBJQvZUJSyBukInqY5N7EJ0/nzsyPQt/c7DcWSgvFuE4sA/ihpjg5ZikISU523gDbtc9AXqYX6wUouhh3LJ1ZI25LwHZQCkFON34Sii0I8wn58/9RtZY8AZ8C86UhbYZbVQvYwQJOYVPprf0rXSiQy4A1sZR4J/jm4oVU6VBtkzvzOBZNME011hmU5yWJ/RB6CPaf5nCO82HEBJorqfQ/QnJZeq3m5Jf87+mnOtitKe/ml8M6a0fcBq6XnGvPoD4o9bzRtrNg478pVxFOU+dbOI/YaV7zAtIWER2V5LRZgDzdmoFXBBxABf3h9PSiX3yPvT3PGbeNHAQctnGqLo1fYhBBKMt2mfxFkq/J5K+7GY8Z/Z1IlxE1qB479mOS+AiNxV6VwljlP5WIA2EkKjRAEvNjwr+xh12V2jzSNzH/IAYZ8R4/VfkmE6MUR8= 2021-09-07T00:26:47.799 INFO:teuthology.kill:2021-09-07 00:26:47,799.799 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-07T00:26:47.802 INFO:teuthology.kill:2021-09-07 00:26:47,802.802 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-07T00:26:47.855 INFO:teuthology.kill:2021-09-07 00:26:47,854.854 INFO:teuthology.orchestra.console:Power off smithi041 2021-09-07T00:26:47.866 INFO:teuthology.kill:2021-09-07 00:26:47,866.866 INFO:teuthology.orchestra.console:Power off smithi052 2021-09-07T00:26:47.995 INFO:teuthology.kill:2021-09-07 00:26:47,994.994 INFO:teuthology.orchestra.console:Power off for smithi041 completed 2021-09-07T00:26:48.004 INFO:teuthology.kill:2021-09-07 00:26:48,004.004 INFO:teuthology.orchestra.console:Power off for smithi052 completed 2021-09-07T00:26:48.136 INFO:teuthology.kill:2021-09-07 00:26:48,135.135 INFO:teuthology.lock.ops:unlocked smithi041.front.sepia.ceph.com 2021-09-07T00:26:48.140 INFO:teuthology.kill:2021-09-07 00:26:48,140.140 INFO:teuthology.lock.ops:unlocked smithi052.front.sepia.ceph.com 2021-09-07T00:28:48.392 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15