2021-06-11T04:18:22.488 INFO:root:teuthology version: 1.1.0-f359b10d 2021-06-11T04:18:22.585 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi124.front.sepia.ceph.com' reimaging 2021-06-11T04:18:22.586 INFO:teuthology.lock.ops:Updating [ubuntu@smithi124.front.sepia.ceph.com]: reset os type and version on server 2021-06-11T04:18:22.586 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2021-06-11T04:18:22.600 INFO:teuthology.lock.ops:Node 'ubuntu@smithi124.front.sepia.ceph.com' reimaging is complete 2021-06-11T04:18:22.600 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi187.front.sepia.ceph.com' reimaging 2021-06-11T04:18:22.600 INFO:teuthology.lock.ops:Updating [ubuntu@smithi187.front.sepia.ceph.com]: reset os type and version on server 2021-06-11T04:18:22.601 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2021-06-11T04:18:22.615 INFO:teuthology.lock.ops:Node 'ubuntu@smithi187.front.sepia.ceph.com' reimaging is complete 2021-06-11T04:18:22.845 INFO:teuthology.provision.fog.smithi124:Scheduling deploy of centos 8.2 2021-06-11T04:18:22.873 INFO:teuthology.provision.fog.smithi187:Scheduling deploy of centos 8.2 2021-06-11T04:18:23.234 INFO:teuthology.orchestra.console:Power off smithi124 2021-06-11T04:18:23.275 INFO:teuthology.orchestra.console:Power off smithi187 2021-06-11T04:18:31.611 INFO:teuthology.orchestra.console:Power off for smithi124 completed 2021-06-11T04:18:31.713 INFO:teuthology.orchestra.console:Power on smithi124 2021-06-11T04:18:35.751 INFO:teuthology.orchestra.console:Power off for smithi187 completed 2021-06-11T04:18:35.854 INFO:teuthology.orchestra.console:Power on smithi187 2021-06-11T04:18:40.075 INFO:teuthology.orchestra.console:Power on for smithi124 completed 2021-06-11T04:18:40.105 INFO:teuthology.orchestra.console:Power on for smithi187 completed 2021-06-11T04:18:40.177 INFO:teuthology.provision.fog.smithi124:Waiting for deploy to finish 2021-06-11T04:18:40.207 INFO:teuthology.provision.fog.smithi187:Waiting for deploy to finish 2021-06-11T04:22:40.861 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-11T04:23:01.089 INFO:teuthology.provision.fog.smithi124:Node is ready 2021-06-11T04:23:01.107 INFO:teuthology.orchestra.run.smithi124.stdout:smithi124.front.sepia.ceph.com 2021-06-11T04:23:01.163 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.15.124 smithi124.front.sepia.ceph.com smithi124 2021-06-11T04:23:01.500 INFO:teuthology.provision.fog.smithi124:Deploy complete! 2021-06-11T04:24:43.997 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-11T04:24:44.300 INFO:teuthology.provision.fog.smithi187:Node is ready 2021-06-11T04:24:44.314 INFO:teuthology.orchestra.run.smithi187.stdout:smithi187.front.sepia.ceph.com 2021-06-11T04:24:44.369 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.15.187 smithi187.front.sepia.ceph.com smithi187 2021-06-11T04:24:44.692 INFO:teuthology.provision.fog.smithi187:Deploy complete! 2021-06-11T04:24:44.807 INFO:teuthology.lock.ops:Checking smithi124.front.sepia.ceph.com 2021-06-11T04:24:44.807 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-11T04:24:44.822 INFO:teuthology.lock.ops:Checking smithi187.front.sepia.ceph.com 2021-06-11T04:24:44.822 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-11T04:24:44.836 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: set os type and version on server 2021-06-11T04:24:44.836 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-11T04:24:45.102 INFO:teuthology.orchestra.run.smithi124.stdout:x86_64 2021-06-11T04:24:45.156 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="CentOS Linux" 2021-06-11T04:24:45.156 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="8 (Core)" 2021-06-11T04:24:45.157 INFO:teuthology.orchestra.run.smithi124.stdout:ID="centos" 2021-06-11T04:24:45.157 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE="rhel fedora" 2021-06-11T04:24:45.157 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="8" 2021-06-11T04:24:45.157 INFO:teuthology.orchestra.run.smithi124.stdout:PLATFORM_ID="platform:el8" 2021-06-11T04:24:45.158 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-06-11T04:24:45.158 INFO:teuthology.orchestra.run.smithi124.stdout:ANSI_COLOR="0;31" 2021-06-11T04:24:45.158 INFO:teuthology.orchestra.run.smithi124.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-06-11T04:24:45.158 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://www.centos.org/" 2021-06-11T04:24:45.159 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-06-11T04:24:45.159 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-06-11T04:24:45.159 INFO:teuthology.orchestra.run.smithi124.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-06-11T04:24:45.159 INFO:teuthology.orchestra.run.smithi124.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-06-11T04:24:45.160 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-06-11T04:24:45.160 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-06-11T04:24:45.160 INFO:teuthology.orchestra.run.smithi124.stdout: 2021-06-11T04:24:45.162 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2021-06-11T04:24:45.178 INFO:teuthology.lock.ops:Updating [smithi187.front.sepia.ceph.com]: set os type and version on server 2021-06-11T04:24:45.179 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-11T04:24:45.445 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2021-06-11T04:24:45.499 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="CentOS Linux" 2021-06-11T04:24:45.499 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="8 (Core)" 2021-06-11T04:24:45.499 INFO:teuthology.orchestra.run.smithi187.stdout:ID="centos" 2021-06-11T04:24:45.499 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="rhel fedora" 2021-06-11T04:24:45.500 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="8" 2021-06-11T04:24:45.500 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el8" 2021-06-11T04:24:45.500 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-06-11T04:24:45.500 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2021-06-11T04:24:45.501 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-06-11T04:24:45.501 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.centos.org/" 2021-06-11T04:24:45.501 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-06-11T04:24:45.501 INFO:teuthology.orchestra.run.smithi187.stdout: 2021-06-11T04:24:45.501 INFO:teuthology.orchestra.run.smithi187.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-06-11T04:24:45.502 INFO:teuthology.orchestra.run.smithi187.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-06-11T04:24:45.502 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-06-11T04:24:45.502 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-06-11T04:24:45.502 INFO:teuthology.orchestra.run.smithi187.stdout: 2021-06-11T04:24:45.503 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2021-06-11T04:24:45.571 INFO:teuthology.dispatcher.supervisor:Running job 6165948 2021-06-11T04:24:45.571 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology -v --owner scheduled_ideepika@teuthology --archive /home/teuthworker/archive/ideepika-2021-06-11_04:16:32-rados-wip-yuri7-testing-2021-06-08-0747-octopus-distro-basic-smithi/6165948 --name ideepika-2021-06-11_04:16:32-rados-wip-yuri7-testing-2021-06-08-0747-octopus-distro-basic-smithi --description rados/cephadm/upgrade/{1-start-distro/1-start-centos_8 2-repo_digest/defaut 3-start-upgrade 4-wait fixed-2} -- /home/teuthworker/archive/ideepika-2021-06-11_04:16:32-rados-wip-yuri7-testing-2021-06-08-0747-octopus-distro-basic-smithi/6165948/orig.config.yaml 2021-06-11T04:24:45.575 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ideepika-2021-06-11_04:16:32-rados-wip-yuri7-testing-2021-06-08-0747-octopus-distro-basic-smithi/6165948 2021-06-11T04:24:45.576 INFO:teuthology.dispatcher.supervisor:Job PID: 889 2021-06-11T04:24:45.576 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-06-11T16:25:31.071 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-06-11T16:25:31.259 INFO:teuthology.kill:Killing Pids: {889} 2021-06-11T16:25:32.287 INFO:teuthology.kill:Nuking machines: ['smithi124', 'smithi187'] 2021-06-11T16:25:33.193 INFO:teuthology.kill:2021-06-11 16:25:33,192.192 INFO:teuthology.nuke:targets: 2021-06-11T16:25:33.193 INFO:teuthology.kill: smithi124.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSQvmpE4jCImYf6RdYWzVXUOHomDQMRNLdtiIt+AJ8hKWgsfWabnL9uUTTzciCGx3+kqj7AYql+oa/4up7XSMOvBaOCAPNTe83LEOZ4kejzTcMq7SDOE98Q6ksea6zSzuXhqNPa9lfQCmeQ+Yax/eEmM7M/bNZ7ZHA/JokzsxLjsOTDWw4hWISVYTj8gwwuKUVJRtleulEhNP5Y2L24HCWx7fXGFw3qtW/KrYn58pX2xkqLiCAGMSB7TpCSyJxivRXLZce8HNsKGrSACEqa7XiDZyTFdMQktSkcl2w0wWsY5AHuvRxaJWOsQNLVTMXy6g/18vV/93L+XwsN4oRmw51y+uGuE7Y4UzJy8U5wi5ZyJYmqTy4H5Wul17n5JKwWYVs5ds6DAC0xaPh8D4y62rq/yluEi8unyjdg7qcD3CmXNzfwPD0501HvfzeXfP+vnHkST0nryMg1a837Xuja1S6REWyEMWTQ8wWH2tXuNPffZt/k0Ejrf3k/p3Wh6g8eHc= 2021-06-11T16:25:33.194 INFO:teuthology.kill: smithi187.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDADwhStcS5oXGk3TYHoKKqyLRb8Pm1fW5T38o/NyjNpgDrHPW09vsIZv0pDMeB/HMqcYjEYmlqoRm2LrRf5ZrBoNiZxYSarGKL/AZEyJiYEhIKdTDrN5TgUkHm7Vm7iyELKA48kfgdHHHKzmCO+tW+RKU/e5FBdgZ3HH4z3ZQIHZfut9Q5uleOqmkVj+IC7kzs6Yqyy/3kBofyVU30ar2npu8GR4/yGgbxv83jYLm5jhUpnVRTfVLWkBrOWmEajLo1+MRlBpjJPaGbwPXV9eZGIWVIVxAs62sJgVCnZxkK4v4mc/PrQEaq8wyGN6FkFIbhoaERryXbeIjD5DCGJ4FtX+t9w5xJu75ljpi15zeXLklEsSnftxCU+JsV7p6PRlCW8nvNzbKTq1GNyuGoaHYPbWr9ag0Ufc7SbEPq27R7LUk3zlyWS7SAchINoE9l5912sEfqQeOv/sRTKAE+L1mdN9GiZazwlw8RGZe+/TcQ7nSqAH0Wlet4YIO3Zx07orU= 2021-06-11T16:25:33.194 INFO:teuthology.kill:2021-06-11 16:25:33,193.193 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-11T16:25:33.195 INFO:teuthology.kill:2021-06-11 16:25:33,195.195 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-11T16:25:33.255 INFO:teuthology.kill:2021-06-11 16:25:33,254.254 INFO:teuthology.orchestra.console:Power off smithi124 2021-06-11T16:25:33.268 INFO:teuthology.kill:2021-06-11 16:25:33,267.267 INFO:teuthology.orchestra.console:Power off smithi187 2021-06-11T16:25:41.621 INFO:teuthology.kill:2021-06-11 16:25:41,621.621 INFO:teuthology.orchestra.console:Power off for smithi124 completed 2021-06-11T16:25:45.739 INFO:teuthology.kill:2021-06-11 16:25:45,738.738 INFO:teuthology.orchestra.console:Power off for smithi187 completed 2021-06-11T16:25:45.961 INFO:teuthology.task.internal:roles: ubuntu@smithi124.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2021-06-11T16:25:45.961 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2021-06-11T16:25:45.962 INFO:teuthology.misc:Compressing logs... 2021-06-11T16:25:45.963 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-11T16:26:46.023 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-06-11T16:26:46.063 INFO:teuthology.kill:No teuthology processes running 2021-06-11T16:26:47.114 INFO:teuthology.kill:Nuking machines: ['smithi124', 'smithi187'] 2021-06-11T16:26:48.093 INFO:teuthology.kill:2021-06-11 16:26:48,093.093 INFO:teuthology.nuke:targets: 2021-06-11T16:26:48.094 INFO:teuthology.kill: smithi124.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSQvmpE4jCImYf6RdYWzVXUOHomDQMRNLdtiIt+AJ8hKWgsfWabnL9uUTTzciCGx3+kqj7AYql+oa/4up7XSMOvBaOCAPNTe83LEOZ4kejzTcMq7SDOE98Q6ksea6zSzuXhqNPa9lfQCmeQ+Yax/eEmM7M/bNZ7ZHA/JokzsxLjsOTDWw4hWISVYTj8gwwuKUVJRtleulEhNP5Y2L24HCWx7fXGFw3qtW/KrYn58pX2xkqLiCAGMSB7TpCSyJxivRXLZce8HNsKGrSACEqa7XiDZyTFdMQktSkcl2w0wWsY5AHuvRxaJWOsQNLVTMXy6g/18vV/93L+XwsN4oRmw51y+uGuE7Y4UzJy8U5wi5ZyJYmqTy4H5Wul17n5JKwWYVs5ds6DAC0xaPh8D4y62rq/yluEi8unyjdg7qcD3CmXNzfwPD0501HvfzeXfP+vnHkST0nryMg1a837Xuja1S6REWyEMWTQ8wWH2tXuNPffZt/k0Ejrf3k/p3Wh6g8eHc= 2021-06-11T16:26:48.095 INFO:teuthology.kill: smithi187.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDADwhStcS5oXGk3TYHoKKqyLRb8Pm1fW5T38o/NyjNpgDrHPW09vsIZv0pDMeB/HMqcYjEYmlqoRm2LrRf5ZrBoNiZxYSarGKL/AZEyJiYEhIKdTDrN5TgUkHm7Vm7iyELKA48kfgdHHHKzmCO+tW+RKU/e5FBdgZ3HH4z3ZQIHZfut9Q5uleOqmkVj+IC7kzs6Yqyy/3kBofyVU30ar2npu8GR4/yGgbxv83jYLm5jhUpnVRTfVLWkBrOWmEajLo1+MRlBpjJPaGbwPXV9eZGIWVIVxAs62sJgVCnZxkK4v4mc/PrQEaq8wyGN6FkFIbhoaERryXbeIjD5DCGJ4FtX+t9w5xJu75ljpi15zeXLklEsSnftxCU+JsV7p6PRlCW8nvNzbKTq1GNyuGoaHYPbWr9ag0Ufc7SbEPq27R7LUk3zlyWS7SAchINoE9l5912sEfqQeOv/sRTKAE+L1mdN9GiZazwlw8RGZe+/TcQ7nSqAH0Wlet4YIO3Zx07orU= 2021-06-11T16:26:48.114 INFO:teuthology.kill:2021-06-11 16:26:48,114.114 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-11T16:26:48.120 INFO:teuthology.kill:2021-06-11 16:26:48,120.120 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-11T16:26:48.166 INFO:teuthology.kill:2021-06-11 16:26:48,166.166 INFO:teuthology.orchestra.console:Power off smithi124 2021-06-11T16:26:48.197 INFO:teuthology.kill:2021-06-11 16:26:48,197.197 INFO:teuthology.orchestra.console:Power off smithi187 2021-06-11T16:26:48.306 INFO:teuthology.kill:2021-06-11 16:26:48,305.305 INFO:teuthology.orchestra.console:Power off for smithi124 completed 2021-06-11T16:26:48.333 INFO:teuthology.kill:2021-06-11 16:26:48,333.333 INFO:teuthology.orchestra.console:Power off for smithi187 completed 2021-06-11T16:26:48.458 INFO:teuthology.kill:2021-06-11 16:26:48,457.457 INFO:teuthology.lock.ops:unlocked smithi124.front.sepia.ceph.com 2021-06-11T16:26:48.469 INFO:teuthology.kill:2021-06-11 16:26:48,468.468 INFO:teuthology.lock.ops:unlocked smithi187.front.sepia.ceph.com 2021-06-11T16:28:48.747 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-06-11T16:28:48.818 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-06-11T16:28:48.819 INFO:teuthology.nuke:Checking targets against current locks 2021-06-11T16:28:48.884 INFO:teuthology.nuke:Not nuking smithi187.front.sepia.ceph.com because description doesn't match 2021-06-11T16:28:48.884 INFO:teuthology.nuke:Not nuking smithi124.front.sepia.ceph.com because description doesn't match