2021-06-15T19:15:15.279 INFO:root:teuthology version: 1.1.0-f359b10d 2021-06-15T19:15:15.378 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi002.front.sepia.ceph.com' reimaging 2021-06-15T19:15:15.380 INFO:teuthology.lock.ops:Updating [ubuntu@smithi002.front.sepia.ceph.com]: reset os type and version on server 2021-06-15T19:15:15.381 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2021-06-15T19:15:15.398 INFO:teuthology.lock.ops:Node 'ubuntu@smithi002.front.sepia.ceph.com' reimaging is complete 2021-06-15T19:15:15.399 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi049.front.sepia.ceph.com' reimaging 2021-06-15T19:15:15.399 INFO:teuthology.lock.ops:Updating [ubuntu@smithi049.front.sepia.ceph.com]: reset os type and version on server 2021-06-15T19:15:15.400 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2021-06-15T19:15:15.417 INFO:teuthology.lock.ops:Node 'ubuntu@smithi049.front.sepia.ceph.com' reimaging is complete 2021-06-15T19:15:15.681 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of centos 8.2 2021-06-15T19:15:15.693 INFO:teuthology.provision.fog.smithi049:Scheduling deploy of centos 8.2 2021-06-15T19:15:16.263 INFO:teuthology.orchestra.console:Power off smithi002 2021-06-15T19:15:16.299 INFO:teuthology.orchestra.console:Power off smithi049 2021-06-15T19:15:28.745 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2021-06-15T19:15:28.769 INFO:teuthology.orchestra.console:Power off for smithi049 completed 2021-06-15T19:15:28.847 INFO:teuthology.orchestra.console:Power on smithi002 2021-06-15T19:15:28.874 INFO:teuthology.orchestra.console:Power on smithi049 2021-06-15T19:15:37.214 INFO:teuthology.orchestra.console:Power on for smithi002 completed 2021-06-15T19:15:37.233 INFO:teuthology.orchestra.console:Power on for smithi049 completed 2021-06-15T19:15:37.315 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish 2021-06-15T19:15:37.333 INFO:teuthology.provision.fog.smithi049:Waiting for deploy to finish 2021-06-15T19:19:35.371 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-15T19:19:45.532 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-15T19:19:45.891 INFO:teuthology.provision.fog.smithi002:Node is ready 2021-06-15T19:19:45.907 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com 2021-06-15T19:19:45.963 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002 2021-06-15T19:19:46.299 INFO:teuthology.provision.fog.smithi002:Deploy complete! 2021-06-15T19:19:50.367 INFO:teuthology.provision.fog.smithi049:Node is ready 2021-06-15T19:19:50.387 INFO:teuthology.orchestra.run.smithi049.stdout:smithi049.front.sepia.ceph.com 2021-06-15T19:19:50.444 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49 smithi049.front.sepia.ceph.com smithi049 2021-06-15T19:19:50.779 INFO:teuthology.provision.fog.smithi049:Deploy complete! 2021-06-15T19:19:50.899 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com 2021-06-15T19:19:50.900 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-15T19:19:50.916 INFO:teuthology.lock.ops:Checking smithi049.front.sepia.ceph.com 2021-06-15T19:19:50.917 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-15T19:19:50.931 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server 2021-06-15T19:19:50.931 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-15T19:19:51.217 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2021-06-15T19:19:51.272 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Linux" 2021-06-15T19:19:51.273 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8 (Core)" 2021-06-15T19:19:51.273 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2021-06-15T19:19:51.274 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2021-06-15T19:19:51.274 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8" 2021-06-15T19:19:51.274 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8" 2021-06-15T19:19:51.275 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-06-15T19:19:51.275 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2021-06-15T19:19:51.275 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-06-15T19:19:51.275 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://www.centos.org/" 2021-06-15T19:19:51.275 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-06-15T19:19:51.276 INFO:teuthology.orchestra.run.smithi002.stdout: 2021-06-15T19:19:51.276 INFO:teuthology.orchestra.run.smithi002.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-06-15T19:19:51.276 INFO:teuthology.orchestra.run.smithi002.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-06-15T19:19:51.276 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-06-15T19:19:51.277 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-06-15T19:19:51.277 INFO:teuthology.orchestra.run.smithi002.stdout: 2021-06-15T19:19:51.279 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2021-06-15T19:19:51.299 INFO:teuthology.lock.ops:Updating [smithi049.front.sepia.ceph.com]: set os type and version on server 2021-06-15T19:19:51.299 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-15T19:19:51.572 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2021-06-15T19:19:51.627 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="CentOS Linux" 2021-06-15T19:19:51.627 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="8 (Core)" 2021-06-15T19:19:51.627 INFO:teuthology.orchestra.run.smithi049.stdout:ID="centos" 2021-06-15T19:19:51.628 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="rhel fedora" 2021-06-15T19:19:51.628 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="8" 2021-06-15T19:19:51.628 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el8" 2021-06-15T19:19:51.628 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-06-15T19:19:51.628 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2021-06-15T19:19:51.629 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-06-15T19:19:51.629 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://www.centos.org/" 2021-06-15T19:19:51.629 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-06-15T19:19:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: 2021-06-15T19:19:51.630 INFO:teuthology.orchestra.run.smithi049.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-06-15T19:19:51.630 INFO:teuthology.orchestra.run.smithi049.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-06-15T19:19:51.630 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-06-15T19:19:51.630 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-06-15T19:19:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: 2021-06-15T19:19:51.631 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2021-06-15T19:19:51.706 INFO:teuthology.dispatcher.supervisor:Running job 6174083 2021-06-15T19:19:51.706 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-06-15_18:44:29-rados-wip-yuri8-testing-2021-06-15-0839-octopus-distro-basic-smithi/6174083 --name yuriw-2021-06-15_18:44:29-rados-wip-yuri8-testing-2021-06-15-0839-octopus-distro-basic-smithi --description rados/cephadm/upgrade/{1-start-distro/1-start-centos_8 2-repo_digest/repo_digest 3-start-upgrade 4-wait fixed-2} -- /home/teuthworker/archive/yuriw-2021-06-15_18:44:29-rados-wip-yuri8-testing-2021-06-15-0839-octopus-distro-basic-smithi/6174083/orig.config.yaml 2021-06-15T19:19:51.710 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-06-15_18:44:29-rados-wip-yuri8-testing-2021-06-15-0839-octopus-distro-basic-smithi/6174083 2021-06-15T19:19:51.711 INFO:teuthology.dispatcher.supervisor:Job PID: 9696 2021-06-15T19:19:51.711 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-06-16T07:20:37.275 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-06-16T07:20:37.443 INFO:teuthology.kill:Killing Pids: {9696} 2021-06-16T07:20:38.410 INFO:teuthology.kill:Nuking machines: ['smithi002', 'smithi049'] 2021-06-16T07:20:39.233 INFO:teuthology.kill:2021-06-16 07:20:39,233.233 INFO:teuthology.nuke:targets: 2021-06-16T07:20:39.234 INFO:teuthology.kill: smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNxWTNCMZbKKHHJBa1BR+T8mtVVpmUG2lOAKe/LVx7wNo7DMV4W+iRHzv6ErdKrhh8zarn/QK1z/FnLajRriv0vu8LkbS4JQy3KzyxgCOYOlgmrQ8b7pAC3/rBJXq5zCkCAidJWRDNerYHQS9lD2BG+QfbWgtbHKJnO4fSMSiZxnUol9hnu09MToi//R8tfvi9NCcnUuIG/veuH+4uwbaZTaDLPMuRhCV33dCRgJiOEzpsR0ZypupKedG52efuwqdP0iqA/QNCghHzfJhEkoZlCL5leqjtvyxwq6tJZI00Jl/o8UnAaSx7O85q4ddnNY8hKYvV9mVHDKpXPekTgbK8WqOucaOb0sZ4lbP+6ATAhqXf9y1vbF/uvmweEqVRWL0GOnqnFxg2tHmo/HfZnsyarfR/ZMMAeD0awsDh9p5Hj4CHD1ZuXlWdZO8XwFCqFiXmrT0hyj7eyzSj0FbPopR9MKk3uyDhnWVTfVrsFfDzJXgilHPKxufsr9I3VQ68KF0= 2021-06-16T07:20:39.234 INFO:teuthology.kill: smithi049.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKbQivUzqSvAIAz37oh2Wq3wLN4TO2zHWRzILZRnzl1NG9goSuaaZtwCA65eqgbLz9s1Ok6zwFyVYSvMcCgv7OzkDZzcjx71ev9yeb/KAn2w0wuq1fFU/wPHDMEiPAsmcWgFUQtwlF+fnixYTvbCxnwxz5dt/XWiqMjhFHyOvvo3jh7gcicm4N5FAqyITpXY+Gs9v3tyTDWxFdYtAN2Yi7q0mh5H8P0PqPk7ln5hT/g9W9U/rRjmQNx1XUnbSpa5bmvmYjJBaTqWTz613MKnceF/S5InOhK2oJBZpacu/ut7Xsu5jmRqHmbb7fZY9qKbn9cWu+BMbcxBUdoNRhAVHb6CCQb19o5mHAISqXLCs73VldYTYm+LViYhrhKHUtQOhvEjdvsWtuDrz/JM+J4eVo2KIxOiSIXK9/LV1iXOxMKymBrw7+b9tiEk8hLvZsyIfrHvzUTzu25H1S8j6yzaxcJPkePskZ/U4R8wc06axGA5+WUXL0ONIPKw3rWMKk1C8= 2021-06-16T07:20:39.235 INFO:teuthology.kill:2021-06-16 07:20:39,233.233 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-16T07:20:39.235 INFO:teuthology.kill:2021-06-16 07:20:39,235.235 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-16T07:20:39.291 INFO:teuthology.kill:2021-06-16 07:20:39,291.291 INFO:teuthology.orchestra.console:Power off smithi049 2021-06-16T07:20:39.305 INFO:teuthology.kill:2021-06-16 07:20:39,305.305 INFO:teuthology.orchestra.console:Power off smithi002 2021-06-16T07:20:47.663 INFO:teuthology.kill:2021-06-16 07:20:47,662.662 INFO:teuthology.orchestra.console:Power off for smithi049 completed 2021-06-16T07:20:47.667 INFO:teuthology.kill:2021-06-16 07:20:47,667.667 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2021-06-16T07:20:47.865 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2021-06-16T07:20:47.865 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2021-06-16T07:20:47.866 INFO:teuthology.misc:Compressing logs... 2021-06-16T07:20:47.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-16T07:21:47.882 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-06-16T07:21:47.924 INFO:teuthology.kill:No teuthology processes running 2021-06-16T07:21:48.814 INFO:teuthology.kill:Nuking machines: ['smithi002', 'smithi049'] 2021-06-16T07:21:49.593 INFO:teuthology.kill:2021-06-16 07:21:49,593.593 INFO:teuthology.nuke:targets: 2021-06-16T07:21:49.594 INFO:teuthology.kill: smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNxWTNCMZbKKHHJBa1BR+T8mtVVpmUG2lOAKe/LVx7wNo7DMV4W+iRHzv6ErdKrhh8zarn/QK1z/FnLajRriv0vu8LkbS4JQy3KzyxgCOYOlgmrQ8b7pAC3/rBJXq5zCkCAidJWRDNerYHQS9lD2BG+QfbWgtbHKJnO4fSMSiZxnUol9hnu09MToi//R8tfvi9NCcnUuIG/veuH+4uwbaZTaDLPMuRhCV33dCRgJiOEzpsR0ZypupKedG52efuwqdP0iqA/QNCghHzfJhEkoZlCL5leqjtvyxwq6tJZI00Jl/o8UnAaSx7O85q4ddnNY8hKYvV9mVHDKpXPekTgbK8WqOucaOb0sZ4lbP+6ATAhqXf9y1vbF/uvmweEqVRWL0GOnqnFxg2tHmo/HfZnsyarfR/ZMMAeD0awsDh9p5Hj4CHD1ZuXlWdZO8XwFCqFiXmrT0hyj7eyzSj0FbPopR9MKk3uyDhnWVTfVrsFfDzJXgilHPKxufsr9I3VQ68KF0= 2021-06-16T07:21:49.594 INFO:teuthology.kill: smithi049.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKbQivUzqSvAIAz37oh2Wq3wLN4TO2zHWRzILZRnzl1NG9goSuaaZtwCA65eqgbLz9s1Ok6zwFyVYSvMcCgv7OzkDZzcjx71ev9yeb/KAn2w0wuq1fFU/wPHDMEiPAsmcWgFUQtwlF+fnixYTvbCxnwxz5dt/XWiqMjhFHyOvvo3jh7gcicm4N5FAqyITpXY+Gs9v3tyTDWxFdYtAN2Yi7q0mh5H8P0PqPk7ln5hT/g9W9U/rRjmQNx1XUnbSpa5bmvmYjJBaTqWTz613MKnceF/S5InOhK2oJBZpacu/ut7Xsu5jmRqHmbb7fZY9qKbn9cWu+BMbcxBUdoNRhAVHb6CCQb19o5mHAISqXLCs73VldYTYm+LViYhrhKHUtQOhvEjdvsWtuDrz/JM+J4eVo2KIxOiSIXK9/LV1iXOxMKymBrw7+b9tiEk8hLvZsyIfrHvzUTzu25H1S8j6yzaxcJPkePskZ/U4R8wc06axGA5+WUXL0ONIPKw3rWMKk1C8= 2021-06-16T07:21:49.611 INFO:teuthology.kill:2021-06-16 07:21:49,611.611 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-16T07:21:49.614 INFO:teuthology.kill:2021-06-16 07:21:49,614.614 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-16T07:21:49.666 INFO:teuthology.kill:2021-06-16 07:21:49,666.666 INFO:teuthology.orchestra.console:Power off smithi049 2021-06-16T07:21:49.677 INFO:teuthology.kill:2021-06-16 07:21:49,677.677 INFO:teuthology.orchestra.console:Power off smithi002 2021-06-16T07:21:49.804 INFO:teuthology.kill:2021-06-16 07:21:49,804.804 INFO:teuthology.orchestra.console:Power off for smithi049 completed 2021-06-16T07:21:49.812 INFO:teuthology.kill:2021-06-16 07:21:49,812.812 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2021-06-16T07:21:49.949 INFO:teuthology.kill:2021-06-16 07:21:49,948.948 INFO:teuthology.lock.ops:unlocked smithi049.front.sepia.ceph.com 2021-06-16T07:21:49.953 INFO:teuthology.kill:2021-06-16 07:21:49,952.952 INFO:teuthology.lock.ops:unlocked smithi002.front.sepia.ceph.com 2021-06-16T07:23:50.204 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15