2021-07-14T21:35:19.273 INFO:root:teuthology version: 1.1.0-22d0fca1
2021-07-14T21:35:20.417 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi023.front.sepia.ceph.com' reimaging
2021-07-14T21:35:20.418 INFO:teuthology.lock.ops:Updating [ubuntu@smithi023.front.sepia.ceph.com]: reset os type and version on server
2021-07-14T21:35:20.418 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server
2021-07-14T21:35:20.434 INFO:teuthology.lock.ops:Node 'ubuntu@smithi023.front.sepia.ceph.com' reimaging is complete
2021-07-14T21:35:20.435 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi035.front.sepia.ceph.com' reimaging
2021-07-14T21:35:20.435 INFO:teuthology.lock.ops:Updating [ubuntu@smithi035.front.sepia.ceph.com]: reset os type and version on server
2021-07-14T21:35:20.436 INFO:teuthology.lock.ops:Updating smithi035.front.sepia.ceph.com on lock server
2021-07-14T21:35:20.459 INFO:teuthology.lock.ops:Node 'ubuntu@smithi035.front.sepia.ceph.com' reimaging is complete
2021-07-14T21:35:20.644 INFO:teuthology.provision.fog.smithi023:Scheduling deploy of centos 8.3
2021-07-14T21:35:20.658 INFO:teuthology.provision.fog.smithi035:Scheduling deploy of centos 8.3
2021-07-14T21:35:21.009 INFO:teuthology.orchestra.console:Power off smithi023
2021-07-14T21:35:21.026 INFO:teuthology.orchestra.console:Power off smithi035
2021-07-14T21:35:33.500 INFO:teuthology.orchestra.console:Power off for smithi023 completed
2021-07-14T21:35:33.511 INFO:teuthology.orchestra.console:Power off for smithi035 completed
2021-07-14T21:35:33.602 INFO:teuthology.orchestra.console:Power on smithi023
2021-07-14T21:35:33.612 INFO:teuthology.orchestra.console:Power on smithi035
2021-07-14T21:35:37.860 INFO:teuthology.orchestra.console:Power on for smithi023 completed
2021-07-14T21:35:37.961 INFO:teuthology.provision.fog.smithi023:Waiting for deploy to finish
2021-07-14T21:35:42.101 INFO:teuthology.orchestra.console:Power on for smithi035 completed
2021-07-14T21:35:42.202 INFO:teuthology.provision.fog.smithi035:Waiting for deploy to finish
2021-07-14T21:39:49.918 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-14T21:39:50.234 INFO:teuthology.provision.fog.smithi035:Node is ready
2021-07-14T21:39:50.250 INFO:teuthology.orchestra.run.smithi035.stdout:smithi035.front.sepia.ceph.com
2021-07-14T21:39:50.306 INFO:teuthology.orchestra.run.smithi035.stdout:172.21.15.35 smithi035.front.sepia.ceph.com smithi035
2021-07-14T21:39:50.715 INFO:teuthology.provision.fog.smithi035:Deploy complete!
2021-07-14T21:40:32.699 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-14T21:40:47.676 INFO:teuthology.provision.fog.smithi023:Node is ready
2021-07-14T21:40:47.696 INFO:teuthology.orchestra.run.smithi023.stdout:smithi023.front.sepia.ceph.com
2021-07-14T21:40:47.754 INFO:teuthology.orchestra.run.smithi023.stdout:172.21.15.23 smithi023.front.sepia.ceph.com smithi023
2021-07-14T21:40:48.108 INFO:teuthology.provision.fog.smithi023:Deploy complete!
2021-07-14T21:40:48.245 INFO:teuthology.lock.ops:Checking smithi023.front.sepia.ceph.com
2021-07-14T21:40:48.246 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-14T21:40:48.267 INFO:teuthology.lock.ops:Checking smithi035.front.sepia.ceph.com
2021-07-14T21:40:48.267 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-14T21:40:48.294 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: set os type and version on server
2021-07-14T21:40:48.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-14T21:40:48.575 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64
2021-07-14T21:40:48.631 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Linux"
2021-07-14T21:40:48.632 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="8"
2021-07-14T21:40:48.632 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos"
2021-07-14T21:40:48.632 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora"
2021-07-14T21:40:48.632 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="8"
2021-07-14T21:40:48.633 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el8"
2021-07-14T21:40:48.633 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Linux 8"
2021-07-14T21:40:48.633 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31"
2021-07-14T21:40:48.633 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-07-14T21:40:48.634 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/"
2021-07-14T21:40:48.634 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-07-14T21:40:48.634 INFO:teuthology.orchestra.run.smithi023.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-07-14T21:40:48.634 INFO:teuthology.orchestra.run.smithi023.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-07-14T21:40:48.636 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server
2021-07-14T21:40:48.654 INFO:teuthology.lock.ops:Updating [smithi035.front.sepia.ceph.com]: set os type and version on server
2021-07-14T21:40:48.654 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-14T21:40:48.953 INFO:teuthology.orchestra.run.smithi035.stdout:x86_64
2021-07-14T21:40:49.008 INFO:teuthology.orchestra.run.smithi035.stdout:NAME="CentOS Linux"
2021-07-14T21:40:49.009 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION="8"
2021-07-14T21:40:49.009 INFO:teuthology.orchestra.run.smithi035.stdout:ID="centos"
2021-07-14T21:40:49.009 INFO:teuthology.orchestra.run.smithi035.stdout:ID_LIKE="rhel fedora"
2021-07-14T21:40:49.010 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION_ID="8"
2021-07-14T21:40:49.010 INFO:teuthology.orchestra.run.smithi035.stdout:PLATFORM_ID="platform:el8"
2021-07-14T21:40:49.010 INFO:teuthology.orchestra.run.smithi035.stdout:PRETTY_NAME="CentOS Linux 8"
2021-07-14T21:40:49.010 INFO:teuthology.orchestra.run.smithi035.stdout:ANSI_COLOR="0;31"
2021-07-14T21:40:49.011 INFO:teuthology.orchestra.run.smithi035.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-07-14T21:40:49.011 INFO:teuthology.orchestra.run.smithi035.stdout:HOME_URL="https://centos.org/"
2021-07-14T21:40:49.011 INFO:teuthology.orchestra.run.smithi035.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-07-14T21:40:49.011 INFO:teuthology.orchestra.run.smithi035.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-07-14T21:40:49.011 INFO:teuthology.orchestra.run.smithi035.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-07-14T21:40:49.012 INFO:teuthology.lock.ops:Updating smithi035.front.sepia.ceph.com on lock server
2021-07-14T21:40:49.091 INFO:teuthology.dispatcher.supervisor:Running job 6272008
2021-07-14T21:40:49.092 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/virtualenv/bin/teuthology -v --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2021-07-14_21:27:59-rados-wip-sage-testing-2021-07-14-1613-distro-basic-smithi/6272008 --name sage-2021-07-14_21:27:59-rados-wip-sage-testing-2021-07-14-1613-distro-basic-smithi --description rados/cephadm/upgrade/{1-start-distro/1-start-centos_8.3-octopus 2-repo_digest/defaut 3-start-upgrade 4-wait mon_election/connectivity} -- /home/teuthworker/archive/sage-2021-07-14_21:27:59-rados-wip-sage-testing-2021-07-14-1613-distro-basic-smithi/6272008/orig.config.yaml
2021-07-14T21:40:49.096 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sage-2021-07-14_21:27:59-rados-wip-sage-testing-2021-07-14-1613-distro-basic-smithi/6272008
2021-07-14T21:40:49.096 INFO:teuthology.dispatcher.supervisor:Job PID: 31853
2021-07-14T21:40:49.097 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-07-15T09:41:42.066 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-07-15T09:41:42.861 INFO:teuthology.kill:Killing Pids: {31853}
2021-07-15T09:41:44.155 INFO:teuthology.kill:Nuking machines: ['smithi023', 'smithi035']
2021-07-15T09:41:45.061 INFO:teuthology.kill:2021-07-15 09:41:45,059.059 INFO:teuthology.nuke:targets:
2021-07-15T09:41:45.062 INFO:teuthology.kill:  smithi023.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjFmDfQfqtBXCORq8phVTX5ZkHwr0Lg3b/7PdNU2PkSbxwssm1n04GZ5JCgiNvXD6SdDzLh+DWn4oK/z2qxu8VIz0EGvsvw5WR2kl6S7ne2gFB+ZUnCi9dWfG68ukg0rd/+Mn0Tl7tS5JS2jPSss0PImeCd/30lM0RD0ZZ+DoN2GoLAuYPZFjTm/SO/mMlMs5xsWCyWFV8ZsBeMfHehfrJlveHjdJedWSRhvquC23nkCHU0s8b9joK7xVLW2M95YT33RpAvJclI5rYiUU4HOY9+44sxkGVFxhE/QpI1xf3eQjGE7VVPFv5DLlmWD+YB4PyQfvEnodW22Fp7gPp6pBy6kbxrOwpSXNgymC6NHwnLI/SBha53jSVxlJAwgvVfAIovnrWiXi3bmOitIRK71Iqqm2JHzvX+xyo6jXyYbQ1b7ZuZyy8QpKWRAiXKAXWEQgwzSGhqH7I7lV8Boqeln18drRufiICJNqDszdLZ0Vf5EoN2SAQbAGlgEsOwgON3l8=
2021-07-15T09:41:45.064 INFO:teuthology.kill:  smithi035.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFcehO1lXB8k2QjJgzW83SNkMWt7O/GUvhJXfGKTfa0iXgowdaFQ8yT1tQXNaNOfkCIXLogMT5ixrYaytDimYLzqu+hoJb3A3gT36H632TD5ohZlsRkA6t+SNTwXoAYDDosI80/T5429aM2eVaJLMKsFl/sdDi4NJ10fjJZj4Lq6ZukgwOBrRZifEo7cOWtJsFjURQw4lbzoCCBZodPUhLCYnHUVQdfGAqen6OBcidCzPi6D5iUi9dJmxlrBZ6ORHV4uebYJ5NaHP/x1HlE8A78VNibiFk1RJcp2rHZyWmNBbejHofQMprgVQJko+meznt8ULDvZ1Sja2NHn2JVRFqveLQFIQGFLrCBtuq0UwMFnvCp/IdxwxuJIsrdARjkRyzqivuPbI7G2u6UC1VCzLbPGAIAOKXtIDmKtp/BXG9t99DK0LM4JbO9qU08OpcuycrQMLOqjUz4Yg+ebr4uS8vdBRpsN56z/QYUGs6hnBY2ULb4cecvbTMv0vjclYvqTE=
2021-07-15T09:41:45.064 INFO:teuthology.kill:2021-07-15 09:41:45,060.060 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-15T09:41:45.065 INFO:teuthology.kill:2021-07-15 09:41:45,062.062 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-15T09:41:45.124 INFO:teuthology.kill:2021-07-15 09:41:45,123.123 INFO:teuthology.orchestra.console:Power off smithi023
2021-07-15T09:41:45.136 INFO:teuthology.kill:2021-07-15 09:41:45,136.136 INFO:teuthology.orchestra.console:Power off smithi035
2021-07-15T09:41:53.503 INFO:teuthology.kill:2021-07-15 09:41:53,502.502 INFO:teuthology.orchestra.console:Power off for smithi035 completed
2021-07-15T09:41:57.610 INFO:teuthology.kill:2021-07-15 09:41:57,609.609 INFO:teuthology.orchestra.console:Power off for smithi023 completed
2021-07-15T09:41:57.880 INFO:teuthology.task.internal:roles: ubuntu@smithi023.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a']
2021-07-15T09:41:57.882 INFO:teuthology.task.internal:roles: ubuntu@smithi035.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a']
2021-07-15T09:41:57.888 INFO:teuthology.misc:Compressing logs...
2021-07-15T09:41:57.904 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-15T09:42:16.539 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/teuthology/orchestra/remote.py", line 509, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-07-15T09:42:16.853 INFO:teuthology.kill:No teuthology processes running
2021-07-15T09:42:17.883 INFO:teuthology.kill:Nuking machines: ['smithi023', 'smithi035']
2021-07-15T09:42:18.777 INFO:teuthology.kill:2021-07-15 09:42:18,777.777 INFO:teuthology.nuke:targets:
2021-07-15T09:42:18.778 INFO:teuthology.kill:  smithi023.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjFmDfQfqtBXCORq8phVTX5ZkHwr0Lg3b/7PdNU2PkSbxwssm1n04GZ5JCgiNvXD6SdDzLh+DWn4oK/z2qxu8VIz0EGvsvw5WR2kl6S7ne2gFB+ZUnCi9dWfG68ukg0rd/+Mn0Tl7tS5JS2jPSss0PImeCd/30lM0RD0ZZ+DoN2GoLAuYPZFjTm/SO/mMlMs5xsWCyWFV8ZsBeMfHehfrJlveHjdJedWSRhvquC23nkCHU0s8b9joK7xVLW2M95YT33RpAvJclI5rYiUU4HOY9+44sxkGVFxhE/QpI1xf3eQjGE7VVPFv5DLlmWD+YB4PyQfvEnodW22Fp7gPp6pBy6kbxrOwpSXNgymC6NHwnLI/SBha53jSVxlJAwgvVfAIovnrWiXi3bmOitIRK71Iqqm2JHzvX+xyo6jXyYbQ1b7ZuZyy8QpKWRAiXKAXWEQgwzSGhqH7I7lV8Boqeln18drRufiICJNqDszdLZ0Vf5EoN2SAQbAGlgEsOwgON3l8=
2021-07-15T09:42:18.779 INFO:teuthology.kill:  smithi035.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFcehO1lXB8k2QjJgzW83SNkMWt7O/GUvhJXfGKTfa0iXgowdaFQ8yT1tQXNaNOfkCIXLogMT5ixrYaytDimYLzqu+hoJb3A3gT36H632TD5ohZlsRkA6t+SNTwXoAYDDosI80/T5429aM2eVaJLMKsFl/sdDi4NJ10fjJZj4Lq6ZukgwOBrRZifEo7cOWtJsFjURQw4lbzoCCBZodPUhLCYnHUVQdfGAqen6OBcidCzPi6D5iUi9dJmxlrBZ6ORHV4uebYJ5NaHP/x1HlE8A78VNibiFk1RJcp2rHZyWmNBbejHofQMprgVQJko+meznt8ULDvZ1Sja2NHn2JVRFqveLQFIQGFLrCBtuq0UwMFnvCp/IdxwxuJIsrdARjkRyzqivuPbI7G2u6UC1VCzLbPGAIAOKXtIDmKtp/BXG9t99DK0LM4JbO9qU08OpcuycrQMLOqjUz4Yg+ebr4uS8vdBRpsN56z/QYUGs6hnBY2ULb4cecvbTMv0vjclYvqTE=
2021-07-15T09:42:18.797 INFO:teuthology.kill:2021-07-15 09:42:18,797.797 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-15T09:42:18.799 INFO:teuthology.kill:2021-07-15 09:42:18,798.798 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-15T09:42:18.851 INFO:teuthology.kill:2021-07-15 09:42:18,851.851 INFO:teuthology.orchestra.console:Power off smithi023
2021-07-15T09:42:18.864 INFO:teuthology.kill:2021-07-15 09:42:18,864.864 INFO:teuthology.orchestra.console:Power off smithi035
2021-07-15T09:42:18.992 INFO:teuthology.kill:2021-07-15 09:42:18,991.991 INFO:teuthology.orchestra.console:Power off for smithi023 completed
2021-07-15T09:42:19.003 INFO:teuthology.kill:2021-07-15 09:42:19,003.003 INFO:teuthology.orchestra.console:Power off for smithi035 completed
2021-07-15T09:42:19.129 INFO:teuthology.kill:2021-07-15 09:42:19,129.129 INFO:teuthology.lock.ops:unlocked smithi023.front.sepia.ceph.com
2021-07-15T09:42:19.141 INFO:teuthology.kill:2021-07-15 09:42:19,140.140 INFO:teuthology.lock.ops:unlocked smithi035.front.sepia.ceph.com
2021-07-15T09:44:19.406 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-07-15T09:44:19.496 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-07-15T09:44:19.498 INFO:teuthology.nuke:Checking targets against current locks
2021-07-15T09:44:19.943 INFO:teuthology.nuke:Not nuking smithi023.front.sepia.ceph.com because description doesn't match
2021-07-15T09:44:19.944 INFO:teuthology.nuke:Not nuking smithi035.front.sepia.ceph.com because description doesn't match