2021-08-24T03:44:07.364 INFO:root:teuthology version: 1.1.0-321319b1 2021-08-24T03:44:07.463 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi036.front.sepia.ceph.com' reimaging 2021-08-24T03:44:07.464 INFO:teuthology.lock.ops:Updating [ubuntu@smithi036.front.sepia.ceph.com]: reset os type and version on server 2021-08-24T03:44:07.465 INFO:teuthology.lock.ops:Updating smithi036.front.sepia.ceph.com on lock server 2021-08-24T03:44:07.481 INFO:teuthology.lock.ops:Node 'ubuntu@smithi036.front.sepia.ceph.com' reimaging is complete 2021-08-24T03:44:07.481 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi053.front.sepia.ceph.com' reimaging 2021-08-24T03:44:07.481 INFO:teuthology.lock.ops:Updating [ubuntu@smithi053.front.sepia.ceph.com]: reset os type and version on server 2021-08-24T03:44:07.482 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2021-08-24T03:44:07.498 INFO:teuthology.lock.ops:Node 'ubuntu@smithi053.front.sepia.ceph.com' reimaging is complete 2021-08-24T03:44:07.707 INFO:teuthology.provision.fog.smithi053:Scheduling deploy of centos 8.2 2021-08-24T03:44:07.718 INFO:teuthology.provision.fog.smithi036:Scheduling deploy of centos 8.2 2021-08-24T03:44:08.189 INFO:teuthology.orchestra.console:Power off smithi036 2021-08-24T03:44:08.205 INFO:teuthology.orchestra.console:Power off smithi053 2021-08-24T03:44:16.568 INFO:teuthology.orchestra.console:Power off for smithi053 completed 2021-08-24T03:44:16.669 INFO:teuthology.orchestra.console:Power on smithi053 2021-08-24T03:44:20.679 INFO:teuthology.orchestra.console:Power off for smithi036 completed 2021-08-24T03:44:20.781 INFO:teuthology.orchestra.console:Power on smithi036 2021-08-24T03:44:25.037 INFO:teuthology.orchestra.console:Power on for smithi036 completed 2021-08-24T03:44:25.042 INFO:teuthology.orchestra.console:Power on for smithi053 completed 2021-08-24T03:44:25.139 INFO:teuthology.provision.fog.smithi036:Waiting for deploy to finish 2021-08-24T03:44:25.143 INFO:teuthology.provision.fog.smithi053:Waiting for deploy to finish 2021-08-24T03:48:33.323 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-08-24T03:48:33.638 INFO:teuthology.provision.fog.smithi053:Node is ready 2021-08-24T03:48:33.694 INFO:teuthology.orchestra.run.smithi053.stdout:smithi053.front.sepia.ceph.com 2021-08-24T03:48:33.751 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53 smithi053.front.sepia.ceph.com smithi053 2021-08-24T03:48:34.078 INFO:teuthology.provision.fog.smithi053:Deploy complete! 2021-08-24T03:48:42.384 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-08-24T03:48:42.692 INFO:teuthology.provision.fog.smithi036:Node is ready 2021-08-24T03:48:42.708 INFO:teuthology.orchestra.run.smithi036.stdout:smithi036.front.sepia.ceph.com 2021-08-24T03:48:42.766 INFO:teuthology.orchestra.run.smithi036.stdout:172.21.15.36 smithi036.front.sepia.ceph.com smithi036 2021-08-24T03:48:43.097 INFO:teuthology.provision.fog.smithi036:Deploy complete! 2021-08-24T03:48:43.235 INFO:teuthology.lock.ops:Checking smithi036.front.sepia.ceph.com 2021-08-24T03:48:43.236 INFO:teuthology.lock.ops:New key found. Updating... 2021-08-24T03:48:43.253 INFO:teuthology.lock.ops:Checking smithi053.front.sepia.ceph.com 2021-08-24T03:48:43.253 INFO:teuthology.lock.ops:New key found. Updating... 2021-08-24T03:48:43.268 INFO:teuthology.lock.ops:Updating [smithi036.front.sepia.ceph.com]: set os type and version on server 2021-08-24T03:48:43.269 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-24T03:48:43.531 INFO:teuthology.orchestra.run.smithi036.stdout:x86_64 2021-08-24T03:48:43.585 INFO:teuthology.orchestra.run.smithi036.stdout:NAME="CentOS Linux" 2021-08-24T03:48:43.585 INFO:teuthology.orchestra.run.smithi036.stdout:VERSION="8 (Core)" 2021-08-24T03:48:43.586 INFO:teuthology.orchestra.run.smithi036.stdout:ID="centos" 2021-08-24T03:48:43.586 INFO:teuthology.orchestra.run.smithi036.stdout:ID_LIKE="rhel fedora" 2021-08-24T03:48:43.586 INFO:teuthology.orchestra.run.smithi036.stdout:VERSION_ID="8" 2021-08-24T03:48:43.586 INFO:teuthology.orchestra.run.smithi036.stdout:PLATFORM_ID="platform:el8" 2021-08-24T03:48:43.587 INFO:teuthology.orchestra.run.smithi036.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-08-24T03:48:43.587 INFO:teuthology.orchestra.run.smithi036.stdout:ANSI_COLOR="0;31" 2021-08-24T03:48:43.587 INFO:teuthology.orchestra.run.smithi036.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-08-24T03:48:43.587 INFO:teuthology.orchestra.run.smithi036.stdout:HOME_URL="https://www.centos.org/" 2021-08-24T03:48:43.587 INFO:teuthology.orchestra.run.smithi036.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-08-24T03:48:43.588 INFO:teuthology.orchestra.run.smithi036.stdout: 2021-08-24T03:48:43.588 INFO:teuthology.orchestra.run.smithi036.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-08-24T03:48:43.588 INFO:teuthology.orchestra.run.smithi036.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-08-24T03:48:43.589 INFO:teuthology.orchestra.run.smithi036.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-08-24T03:48:43.589 INFO:teuthology.orchestra.run.smithi036.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-08-24T03:48:43.589 INFO:teuthology.orchestra.run.smithi036.stdout: 2021-08-24T03:48:43.591 INFO:teuthology.lock.ops:Updating smithi036.front.sepia.ceph.com on lock server 2021-08-24T03:48:43.607 INFO:teuthology.lock.ops:Updating [smithi053.front.sepia.ceph.com]: set os type and version on server 2021-08-24T03:48:43.607 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-24T03:48:43.881 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2021-08-24T03:48:43.938 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Linux" 2021-08-24T03:48:43.938 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="8 (Core)" 2021-08-24T03:48:43.938 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2021-08-24T03:48:43.938 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2021-08-24T03:48:43.939 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="8" 2021-08-24T03:48:43.939 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el8" 2021-08-24T03:48:43.939 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-08-24T03:48:43.939 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2021-08-24T03:48:43.939 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-08-24T03:48:43.940 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.centos.org/" 2021-08-24T03:48:43.940 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-08-24T03:48:43.940 INFO:teuthology.orchestra.run.smithi053.stdout: 2021-08-24T03:48:43.940 INFO:teuthology.orchestra.run.smithi053.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-08-24T03:48:43.941 INFO:teuthology.orchestra.run.smithi053.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-08-24T03:48:43.941 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-08-24T03:48:43.941 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-08-24T03:48:43.941 INFO:teuthology.orchestra.run.smithi053.stdout: 2021-08-24T03:48:43.942 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2021-08-24T03:48:44.027 INFO:teuthology.dispatcher.supervisor:Running job 6353851 2021-08-24T03:48:44.027 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-08-23_19:24:05-rados-wip-yuri4-testing-2021-08-23-0812-pacific-distro-basic-smithi/6353851 --name yuriw-2021-08-23_19:24:05-rados-wip-yuri4-testing-2021-08-23-0812-pacific-distro-basic-smithi --description rados/mgr/{clusters/{2-node-mgr} debug/mgr mon_election/classic objectstore/bluestore-hybrid supported-random-distro$/{centos_8} tasks/progress} -- /home/teuthworker/archive/yuriw-2021-08-23_19:24:05-rados-wip-yuri4-testing-2021-08-23-0812-pacific-distro-basic-smithi/6353851/orig.config.yaml 2021-08-24T03:48:44.031 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-08-23_19:24:05-rados-wip-yuri4-testing-2021-08-23-0812-pacific-distro-basic-smithi/6353851 2021-08-24T03:48:44.032 INFO:teuthology.dispatcher.supervisor:Job PID: 22005 2021-08-24T03:48:44.032 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-08-24T15:49:36.009 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-08-24T15:49:36.183 INFO:teuthology.kill:Killing Pids: {22005} 2021-08-24T15:49:37.207 INFO:teuthology.kill:Nuking machines: ['smithi036', 'smithi053'] 2021-08-24T15:49:38.052 INFO:teuthology.kill:2021-08-24 15:49:38,052.052 INFO:teuthology.nuke:targets: 2021-08-24T15:49:38.053 INFO:teuthology.kill: smithi036.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC93D75t4do2oq1ws3sSrCXhpBldcL7LXcrEJi8tBrlbfrZ2Q6tRht7OBpJPc1zobwT8X8Oob9iE92QToE7lnUTK2SJOtvnvGNH1Id0/Za5bGi1V4cHytYfL7NcqQXcYbcz0hn1g5MeLY/6daSP2fzbB6KwkqvabXFB4USrqVglKNloIHrmuX3TbDWFDLelSfpwWvM6ExmBURGYed6n0ddQHCJcv0qVPR3QnyBMvTzql5+hmsoz6MrcEB9txik6pRSw8Bi3gQfC14WOoqM4e6D5pknzzB66dUrLSk8qZHBT4iHJw5k/XbygNm6MQA5BPyFQpS77UhZACNe7/FQcecqIRzds4fQws56YjdMTzALiyn1xHxnKomyei8BxUSa0RUckKpEZZUGdlHqvaj2nQejkvFq0pLw1nMLj3kYGWIwLmdpboC7Kcas4pS0W1/zoZe1AIwMBYkagQc4TZ2HBEot6yMOisfMj+nh34r+JzItPTLsAbFpOkGjuN7LYJHed758= 2021-08-24T15:49:38.053 INFO:teuthology.kill: smithi053.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDp3F4fCaQ9Ueku30de+moWxcYRgfxtzDYaDdW4PGXglqZ3goKtPwURr/wAdmwMcfZd4NRBoZ/b4AaudvH9C5Pu7+ZcGweDwiuEm3tg6FVgHlcrXnQqEk9HqT5TtNOsN9Kkvr3Qg+NZpLWGY4Nvk6sqJSqknAQ2bxK5IdM0Rn0rLsuyqvbedSfNKfpVt86xh8VbPKUS1vj0s8mkbJNWoET9JoSjSV4UGJpuGjeYtm9kUrTrfhXYin3B0mfzf+P83doZ31nrECRamw097ezBbix1h2np5zNhsOssMD6EULTe1etKdmtWTCvQMm5CchAcjeb8QL0XEsnaRDTmmYkWZcsY6U8o4XxRojnlhA5OisfYstBJi80m67yog/5hv+UQYmR4tCyIU/PgFlqFvJdh6XR45E+IMMD/aIJ1kJSDJZRFVhdAsWGIvCqyp7J8YQl9kcu9pUhk5RhNDiHYPVxtIYjZkxWLMAl8uo4a9NyeKWMrAaOhsvnG7JChHMXqe3RZ7fE= 2021-08-24T15:49:38.053 INFO:teuthology.kill:2021-08-24 15:49:38,052.052 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-24T15:49:38.054 INFO:teuthology.kill:2021-08-24 15:49:38,054.054 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-24T15:49:38.113 INFO:teuthology.kill:2021-08-24 15:49:38,112.112 INFO:teuthology.orchestra.console:Power off smithi036 2021-08-24T15:49:38.125 INFO:teuthology.kill:2021-08-24 15:49:38,125.125 INFO:teuthology.orchestra.console:Power off smithi053 2021-08-24T15:49:46.488 INFO:teuthology.kill:2021-08-24 15:49:46,487.487 INFO:teuthology.orchestra.console:Power off for smithi053 completed 2021-08-24T15:49:50.593 INFO:teuthology.kill:2021-08-24 15:49:50,593.593 INFO:teuthology.orchestra.console:Power off for smithi036 completed 2021-08-24T15:49:50.819 INFO:teuthology.task.internal:roles: ubuntu@smithi036.front.sepia.ceph.com - ['mgr.x', 'mon.a', 'mon.c', 'mds.a', 'mds.c', 'osd.0', 'client.0'] 2021-08-24T15:49:50.820 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mgr.y', 'mgr.z', 'mon.b', 'mds.b', 'osd.1', 'osd.2', 'osd.3', 'client.1'] 2021-08-24T15:49:50.820 INFO:teuthology.misc:Compressing logs... 2021-08-24T15:49:50.821 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-24T15:50:09.355 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-08-24T15:50:09.401 INFO:teuthology.kill:No teuthology processes running 2021-08-24T15:50:10.400 INFO:teuthology.kill:Nuking machines: ['smithi036', 'smithi053'] 2021-08-24T15:50:11.299 INFO:teuthology.kill:2021-08-24 15:50:11,298.298 INFO:teuthology.nuke:targets: 2021-08-24T15:50:11.299 INFO:teuthology.kill: smithi036.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC93D75t4do2oq1ws3sSrCXhpBldcL7LXcrEJi8tBrlbfrZ2Q6tRht7OBpJPc1zobwT8X8Oob9iE92QToE7lnUTK2SJOtvnvGNH1Id0/Za5bGi1V4cHytYfL7NcqQXcYbcz0hn1g5MeLY/6daSP2fzbB6KwkqvabXFB4USrqVglKNloIHrmuX3TbDWFDLelSfpwWvM6ExmBURGYed6n0ddQHCJcv0qVPR3QnyBMvTzql5+hmsoz6MrcEB9txik6pRSw8Bi3gQfC14WOoqM4e6D5pknzzB66dUrLSk8qZHBT4iHJw5k/XbygNm6MQA5BPyFQpS77UhZACNe7/FQcecqIRzds4fQws56YjdMTzALiyn1xHxnKomyei8BxUSa0RUckKpEZZUGdlHqvaj2nQejkvFq0pLw1nMLj3kYGWIwLmdpboC7Kcas4pS0W1/zoZe1AIwMBYkagQc4TZ2HBEot6yMOisfMj+nh34r+JzItPTLsAbFpOkGjuN7LYJHed758= 2021-08-24T15:50:11.300 INFO:teuthology.kill: smithi053.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDp3F4fCaQ9Ueku30de+moWxcYRgfxtzDYaDdW4PGXglqZ3goKtPwURr/wAdmwMcfZd4NRBoZ/b4AaudvH9C5Pu7+ZcGweDwiuEm3tg6FVgHlcrXnQqEk9HqT5TtNOsN9Kkvr3Qg+NZpLWGY4Nvk6sqJSqknAQ2bxK5IdM0Rn0rLsuyqvbedSfNKfpVt86xh8VbPKUS1vj0s8mkbJNWoET9JoSjSV4UGJpuGjeYtm9kUrTrfhXYin3B0mfzf+P83doZ31nrECRamw097ezBbix1h2np5zNhsOssMD6EULTe1etKdmtWTCvQMm5CchAcjeb8QL0XEsnaRDTmmYkWZcsY6U8o4XxRojnlhA5OisfYstBJi80m67yog/5hv+UQYmR4tCyIU/PgFlqFvJdh6XR45E+IMMD/aIJ1kJSDJZRFVhdAsWGIvCqyp7J8YQl9kcu9pUhk5RhNDiHYPVxtIYjZkxWLMAl8uo4a9NyeKWMrAaOhsvnG7JChHMXqe3RZ7fE= 2021-08-24T15:50:11.319 INFO:teuthology.kill:2021-08-24 15:50:11,319.319 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-24T15:50:11.322 INFO:teuthology.kill:2021-08-24 15:50:11,322.322 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-24T15:50:11.376 INFO:teuthology.kill:2021-08-24 15:50:11,376.376 INFO:teuthology.orchestra.console:Power off smithi053 2021-08-24T15:50:11.389 INFO:teuthology.kill:2021-08-24 15:50:11,388.388 INFO:teuthology.orchestra.console:Power off smithi036 2021-08-24T15:50:11.515 INFO:teuthology.kill:2021-08-24 15:50:11,515.515 INFO:teuthology.orchestra.console:Power off for smithi053 completed 2021-08-24T15:50:11.528 INFO:teuthology.kill:2021-08-24 15:50:11,528.528 INFO:teuthology.orchestra.console:Power off for smithi036 completed 2021-08-24T15:50:11.657 INFO:teuthology.kill:2021-08-24 15:50:11,656.656 INFO:teuthology.lock.ops:unlocked smithi053.front.sepia.ceph.com 2021-08-24T15:50:11.664 INFO:teuthology.kill:2021-08-24 15:50:11,663.663 INFO:teuthology.lock.ops:unlocked smithi036.front.sepia.ceph.com 2021-08-24T15:52:11.841 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15