2021-03-17T13:42:23.887 INFO:root:teuthology version: 1.1.0-b9656917 2021-03-17T13:42:23.976 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi091.front.sepia.ceph.com' reimaging 2021-03-17T13:42:23.977 INFO:teuthology.lock.ops:Updating [ubuntu@smithi091.front.sepia.ceph.com]: reset os type and version on server 2021-03-17T13:42:23.977 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server 2021-03-17T13:42:23.993 INFO:teuthology.lock.ops:Node 'ubuntu@smithi091.front.sepia.ceph.com' reimaging is complete 2021-03-17T13:42:23.994 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging 2021-03-17T13:42:23.994 INFO:teuthology.lock.ops:Updating [ubuntu@smithi116.front.sepia.ceph.com]: reset os type and version on server 2021-03-17T13:42:23.994 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2021-03-17T13:42:24.010 INFO:teuthology.lock.ops:Node 'ubuntu@smithi116.front.sepia.ceph.com' reimaging is complete 2021-03-17T13:42:24.250 INFO:teuthology.provision.fog.smithi091:Scheduling deploy of centos 8.2 2021-03-17T13:42:24.291 INFO:teuthology.provision.fog.smithi116:Scheduling deploy of centos 8.2 2021-03-17T13:42:24.905 INFO:teuthology.orchestra.console:Power off smithi091 2021-03-17T13:42:24.917 INFO:teuthology.orchestra.console:Power off smithi116 2021-03-17T13:42:33.267 INFO:teuthology.orchestra.console:Power off for smithi091 completed 2021-03-17T13:42:33.273 INFO:teuthology.orchestra.console:Power off for smithi116 completed 2021-03-17T13:42:33.369 INFO:teuthology.orchestra.console:Power on smithi091 2021-03-17T13:42:33.376 INFO:teuthology.orchestra.console:Power on smithi116 2021-03-17T13:42:41.731 INFO:teuthology.orchestra.console:Power on for smithi091 completed 2021-03-17T13:42:41.737 INFO:teuthology.orchestra.console:Power on for smithi116 completed 2021-03-17T13:42:41.832 INFO:teuthology.provision.fog.smithi091:Waiting for deploy to finish 2021-03-17T13:42:41.838 INFO:teuthology.provision.fog.smithi116:Waiting for deploy to finish 2021-03-17T13:47:59.336 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-17T13:48:19.931 INFO:teuthology.provision.fog.smithi091:Node is ready 2021-03-17T13:48:19.949 INFO:teuthology.orchestra.run.smithi091.stdout:smithi091.front.sepia.ceph.com 2021-03-17T13:48:20.006 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.15.91 smithi091.front.sepia.ceph.com smithi091 2021-03-17T13:48:20.350 INFO:teuthology.provision.fog.smithi091:Deploy complete! 2021-03-17T13:48:40.033 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-17T13:48:40.338 INFO:teuthology.provision.fog.smithi116:Node is ready 2021-03-17T13:48:40.352 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116.front.sepia.ceph.com 2021-03-17T13:48:40.406 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116 smithi116.front.sepia.ceph.com smithi116 2021-03-17T13:48:40.720 INFO:teuthology.provision.fog.smithi116:Deploy complete! 2021-03-17T13:48:40.842 INFO:teuthology.lock.ops:Checking smithi091.front.sepia.ceph.com 2021-03-17T13:48:40.842 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-17T13:48:40.872 INFO:teuthology.lock.ops:Checking smithi116.front.sepia.ceph.com 2021-03-17T13:48:40.872 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-17T13:48:40.949 INFO:teuthology.lock.ops:Updating [smithi091.front.sepia.ceph.com]: set os type and version on server 2021-03-17T13:48:40.949 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-17T13:48:41.220 INFO:teuthology.orchestra.run.smithi091.stdout:x86_64 2021-03-17T13:48:41.276 INFO:teuthology.orchestra.run.smithi091.stdout:NAME="CentOS Linux" 2021-03-17T13:48:41.276 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION="8 (Core)" 2021-03-17T13:48:41.276 INFO:teuthology.orchestra.run.smithi091.stdout:ID="centos" 2021-03-17T13:48:41.277 INFO:teuthology.orchestra.run.smithi091.stdout:ID_LIKE="rhel fedora" 2021-03-17T13:48:41.277 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_ID="8" 2021-03-17T13:48:41.277 INFO:teuthology.orchestra.run.smithi091.stdout:PLATFORM_ID="platform:el8" 2021-03-17T13:48:41.277 INFO:teuthology.orchestra.run.smithi091.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-03-17T13:48:41.277 INFO:teuthology.orchestra.run.smithi091.stdout:ANSI_COLOR="0;31" 2021-03-17T13:48:41.277 INFO:teuthology.orchestra.run.smithi091.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-03-17T13:48:41.277 INFO:teuthology.orchestra.run.smithi091.stdout:HOME_URL="https://www.centos.org/" 2021-03-17T13:48:41.277 INFO:teuthology.orchestra.run.smithi091.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-03-17T13:48:41.277 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-03-17T13:48:41.277 INFO:teuthology.orchestra.run.smithi091.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-03-17T13:48:41.278 INFO:teuthology.orchestra.run.smithi091.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-03-17T13:48:41.278 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-03-17T13:48:41.278 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-03-17T13:48:41.278 INFO:teuthology.orchestra.run.smithi091.stdout: 2021-03-17T13:48:41.279 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server 2021-03-17T13:48:41.304 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: set os type and version on server 2021-03-17T13:48:41.304 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-17T13:48:41.628 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2021-03-17T13:48:41.682 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="CentOS Linux" 2021-03-17T13:48:41.683 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="8 (Core)" 2021-03-17T13:48:41.683 INFO:teuthology.orchestra.run.smithi116.stdout:ID="centos" 2021-03-17T13:48:41.683 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="rhel fedora" 2021-03-17T13:48:41.683 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="8" 2021-03-17T13:48:41.684 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el8" 2021-03-17T13:48:41.684 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-03-17T13:48:41.684 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31" 2021-03-17T13:48:41.684 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-03-17T13:48:41.684 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://www.centos.org/" 2021-03-17T13:48:41.685 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-03-17T13:48:41.685 INFO:teuthology.orchestra.run.smithi116.stdout: 2021-03-17T13:48:41.685 INFO:teuthology.orchestra.run.smithi116.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-03-17T13:48:41.685 INFO:teuthology.orchestra.run.smithi116.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-03-17T13:48:41.685 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-03-17T13:48:41.685 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-03-17T13:48:41.686 INFO:teuthology.orchestra.run.smithi116.stdout: 2021-03-17T13:48:41.687 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2021-03-17T13:48:41.831 INFO:teuthology.dispatcher.supervisor:Running job 5973819 2021-03-17T13:48:41.831 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/bin/teuthology -v --owner scheduled_ideepika@teuthology --archive /home/teuthworker/archive/ideepika-2021-03-17_08:00:31-rados-wip-tracker-48613-distro-basic-smithi/5973819 --name ideepika-2021-03-17_08:00:31-rados-wip-tracker-48613-distro-basic-smithi --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/valgrind} -- /home/teuthworker/archive/ideepika-2021-03-17_08:00:31-rados-wip-tracker-48613-distro-basic-smithi/5973819/orig.config.yaml 2021-03-17T13:48:41.836 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ideepika-2021-03-17_08:00:31-rados-wip-tracker-48613-distro-basic-smithi/5973819 2021-03-17T13:48:41.836 INFO:teuthology.dispatcher.supervisor:Job PID: 30207 2021-03-17T13:48:41.836 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-03-18T01:49:31.231 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-03-18T01:49:31.722 INFO:teuthology.kill:Killing Pids: {30207} 2021-03-18T01:49:32.764 INFO:teuthology.kill:Nuking machines: ['smithi091', 'smithi116'] 2021-03-18T01:49:33.680 INFO:teuthology.kill:2021-03-18 01:49:33,680.680 INFO:teuthology.nuke:targets: 2021-03-18T01:49:33.681 INFO:teuthology.kill: smithi091.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxGew2Q78yYcJWVLANw/AA5Yd12IQFrrGWOdy47hvPb7iU+7oa2Orv1veVZQvaiGRgk8PpZGbo49zZsEp4WOVzMg4L5poMqyuCfKd3yzTaRmTEMpGwzddv0XxChTEh2SIvzsfsW0Quwoajmpllcm8MwE7gbge3w3T28kIEP+0vfYxAhXhPNpb5+V3SFv79Mm8yTtX+mzCmKn+E+UEhV2c7HWpTLysPelXhKvWMfPrWKLeKIsQHO0WZS6/wo2PnOqsC8b8dRxFqTSmlu7qb3+k4+5M29ftQiyCKup7DK0YBO1Zsfx63mvTbXDUHSf/z+E+a1n0j+fQRuB3Cija/ggIWBuJ/cpfoVPhbgS5IfNxQeDTzoSUUwArNZojKv/dsc9xcRad9LdYCkBq3SHtVPR4T2lncCFsvNnT7WX6dI78rqMOYtDY8WopYaAfcfT4TfmTb1wOXJSbBxnPrTAp4/pw0mwJeXU26pmwOZ6Zqbqc+1hHrhV3sA0BB7f+W6QgujKs= 2021-03-18T01:49:33.681 INFO:teuthology.kill: smithi116.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChLKvcLqqWftGQueCf7tusDStcws4OI7A+oOFBxI3Ba+WlvpItONP3DZP4Vw7plNwvdescd+fNP0WzcKZJa4UrZMsfLn6/woPeIjLWEnjzuj96dkH1U4/XZqbZejjP/GzO3LszjN0TM/pAkw9wqgPIV3PNCrbhxD0j4l0f+RC2OuH7TJqfJ309a2nJEBWAR+Q0NbTkvQ4Apo3WqwrRZyNadTksoRM2Not2kjEHO02/g/txgxLSKZbK6dTRJem0597YYRMbpbe0bHWAhfos8yP5fFm/m7JCVltK0arE28kwYuk4CBJwehbuhLa+CgBS0Ms/vQEWn0YSy1Py1PhQreP9S6k1QbcJt6Nvxuko4jJz26hgfyfJos3Lljbw/FlmPafDZFjn7utC+m5fX7ik68e9TFoCK14HRT4OwIVemrUcf/yKSmnGt0nmVxe78BWug5nnhMEVm6e7/+vVAW2MRYsHlUYpLX0e9sMIntEodRSJWaNJ4JH/CtNuqIa0xGyk8JE= 2021-03-18T01:49:33.681 INFO:teuthology.kill:2021-03-18 01:49:33,681.681 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-18T01:49:33.683 INFO:teuthology.kill:2021-03-18 01:49:33,683.683 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-18T01:49:33.746 INFO:teuthology.kill:2021-03-18 01:49:33,745.745 INFO:teuthology.orchestra.console:Power off smithi091 2021-03-18T01:49:33.757 INFO:teuthology.kill:2021-03-18 01:49:33,757.757 INFO:teuthology.orchestra.console:Power off smithi116 2021-03-18T01:49:42.114 INFO:teuthology.kill:2021-03-18 01:49:42,113.113 INFO:teuthology.orchestra.console:Power off for smithi116 completed 2021-03-18T01:49:46.224 INFO:teuthology.kill:2021-03-18 01:49:46,223.223 INFO:teuthology.orchestra.console:Power off for smithi091 completed 2021-03-18T01:49:46.461 INFO:teuthology.task.internal:roles: ubuntu@smithi091.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-03-18T01:49:46.462 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-03-18T01:49:46.462 INFO:teuthology.misc:Compressing logs... 2021-03-18T01:49:46.462 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-18T01:50:46.522 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/misc.py", line 1374, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-03-18T01:50:46.586 INFO:teuthology.kill:No teuthology processes running 2021-03-18T01:50:47.682 INFO:teuthology.kill:Nuking machines: ['smithi091', 'smithi116'] 2021-03-18T01:50:48.593 INFO:teuthology.kill:2021-03-18 01:50:48,592.592 INFO:teuthology.nuke:targets: 2021-03-18T01:50:48.593 INFO:teuthology.kill: smithi091.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxGew2Q78yYcJWVLANw/AA5Yd12IQFrrGWOdy47hvPb7iU+7oa2Orv1veVZQvaiGRgk8PpZGbo49zZsEp4WOVzMg4L5poMqyuCfKd3yzTaRmTEMpGwzddv0XxChTEh2SIvzsfsW0Quwoajmpllcm8MwE7gbge3w3T28kIEP+0vfYxAhXhPNpb5+V3SFv79Mm8yTtX+mzCmKn+E+UEhV2c7HWpTLysPelXhKvWMfPrWKLeKIsQHO0WZS6/wo2PnOqsC8b8dRxFqTSmlu7qb3+k4+5M29ftQiyCKup7DK0YBO1Zsfx63mvTbXDUHSf/z+E+a1n0j+fQRuB3Cija/ggIWBuJ/cpfoVPhbgS5IfNxQeDTzoSUUwArNZojKv/dsc9xcRad9LdYCkBq3SHtVPR4T2lncCFsvNnT7WX6dI78rqMOYtDY8WopYaAfcfT4TfmTb1wOXJSbBxnPrTAp4/pw0mwJeXU26pmwOZ6Zqbqc+1hHrhV3sA0BB7f+W6QgujKs= 2021-03-18T01:50:48.593 INFO:teuthology.kill: smithi116.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChLKvcLqqWftGQueCf7tusDStcws4OI7A+oOFBxI3Ba+WlvpItONP3DZP4Vw7plNwvdescd+fNP0WzcKZJa4UrZMsfLn6/woPeIjLWEnjzuj96dkH1U4/XZqbZejjP/GzO3LszjN0TM/pAkw9wqgPIV3PNCrbhxD0j4l0f+RC2OuH7TJqfJ309a2nJEBWAR+Q0NbTkvQ4Apo3WqwrRZyNadTksoRM2Not2kjEHO02/g/txgxLSKZbK6dTRJem0597YYRMbpbe0bHWAhfos8yP5fFm/m7JCVltK0arE28kwYuk4CBJwehbuhLa+CgBS0Ms/vQEWn0YSy1Py1PhQreP9S6k1QbcJt6Nvxuko4jJz26hgfyfJos3Lljbw/FlmPafDZFjn7utC+m5fX7ik68e9TFoCK14HRT4OwIVemrUcf/yKSmnGt0nmVxe78BWug5nnhMEVm6e7/+vVAW2MRYsHlUYpLX0e9sMIntEodRSJWaNJ4JH/CtNuqIa0xGyk8JE= 2021-03-18T01:50:48.614 INFO:teuthology.kill:2021-03-18 01:50:48,614.614 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-18T01:50:48.617 INFO:teuthology.kill:2021-03-18 01:50:48,616.616 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-18T01:50:48.676 INFO:teuthology.kill:2021-03-18 01:50:48,675.675 INFO:teuthology.orchestra.console:Power off smithi091 2021-03-18T01:50:48.687 INFO:teuthology.kill:2021-03-18 01:50:48,687.687 INFO:teuthology.orchestra.console:Power off smithi116 2021-03-18T01:50:48.815 INFO:teuthology.kill:2021-03-18 01:50:48,815.815 INFO:teuthology.orchestra.console:Power off for smithi091 completed 2021-03-18T01:50:48.825 INFO:teuthology.kill:2021-03-18 01:50:48,825.825 INFO:teuthology.orchestra.console:Power off for smithi116 completed 2021-03-18T01:50:48.959 INFO:teuthology.kill:2021-03-18 01:50:48,958.958 INFO:teuthology.lock.ops:unlocked smithi091.front.sepia.ceph.com 2021-03-18T01:50:48.963 INFO:teuthology.kill:2021-03-18 01:50:48,963.963 INFO:teuthology.lock.ops:unlocked smithi116.front.sepia.ceph.com 2021-03-18T01:52:49.214 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-03-18T01:52:49.304 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-03-18T01:52:49.304 INFO:teuthology.nuke:Checking targets against current locks 2021-03-18T01:52:49.379 INFO:teuthology.nuke:Not nuking smithi116.front.sepia.ceph.com because description doesn't match 2021-03-18T01:52:49.379 INFO:teuthology.nuke:Not nuking smithi091.front.sepia.ceph.com because description doesn't match