2021-07-14T21:30:24.142 INFO:root:teuthology version: 1.1.0-22d0fca1
2021-07-14T21:30:24.388 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi059.front.sepia.ceph.com' reimaging
2021-07-14T21:30:24.389 INFO:teuthology.lock.ops:Updating [ubuntu@smithi059.front.sepia.ceph.com]: reset os type and version on server
2021-07-14T21:30:24.396 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server
2021-07-14T21:30:24.418 INFO:teuthology.lock.ops:Node 'ubuntu@smithi059.front.sepia.ceph.com' reimaging is complete
2021-07-14T21:30:24.418 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi103.front.sepia.ceph.com' reimaging
2021-07-14T21:30:24.418 INFO:teuthology.lock.ops:Updating [ubuntu@smithi103.front.sepia.ceph.com]: reset os type and version on server
2021-07-14T21:30:24.419 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server
2021-07-14T21:30:24.446 INFO:teuthology.lock.ops:Node 'ubuntu@smithi103.front.sepia.ceph.com' reimaging is complete
2021-07-14T21:30:24.671 INFO:teuthology.provision.fog.smithi059:Scheduling deploy of rhel 8.3
2021-07-14T21:30:24.680 INFO:teuthology.provision.fog.smithi103:Scheduling deploy of rhel 8.3
2021-07-14T21:30:25.208 INFO:teuthology.orchestra.console:Power off smithi103
2021-07-14T21:30:25.267 INFO:teuthology.orchestra.console:Power off smithi059
2021-07-14T21:30:37.926 INFO:teuthology.orchestra.console:Power off for smithi103 completed
2021-07-14T21:30:37.950 INFO:teuthology.orchestra.console:Power off for smithi059 completed
2021-07-14T21:30:38.055 INFO:teuthology.orchestra.console:Power on smithi103
2021-07-14T21:30:38.070 INFO:teuthology.orchestra.console:Power on smithi059
2021-07-14T21:30:46.576 INFO:teuthology.orchestra.console:Power on for smithi103 completed
2021-07-14T21:30:46.632 INFO:teuthology.orchestra.console:Power on for smithi059 completed
2021-07-14T21:30:46.678 INFO:teuthology.provision.fog.smithi103:Waiting for deploy to finish
2021-07-14T21:30:46.739 INFO:teuthology.provision.fog.smithi059:Waiting for deploy to finish
2021-07-14T21:34:54.463 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-14T21:34:54.774 INFO:teuthology.provision.fog.smithi059:Node is ready
2021-07-14T21:34:54.792 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059.front.sepia.ceph.com
2021-07-14T21:34:54.847 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59 smithi059.front.sepia.ceph.com smithi059
2021-07-14T21:34:55.180 INFO:teuthology.provision.fog.smithi059:Deploy complete!
2021-07-14T21:35:26.436 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-14T21:35:47.107 INFO:teuthology.provision.fog.smithi103:Node is ready
2021-07-14T21:35:47.127 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103.front.sepia.ceph.com
2021-07-14T21:35:47.184 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103 smithi103.front.sepia.ceph.com smithi103
2021-07-14T21:35:47.546 INFO:teuthology.provision.fog.smithi103:Deploy complete!
2021-07-14T21:35:47.680 INFO:teuthology.lock.ops:Checking smithi059.front.sepia.ceph.com
2021-07-14T21:35:47.681 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-14T21:35:47.699 INFO:teuthology.lock.ops:Checking smithi103.front.sepia.ceph.com
2021-07-14T21:35:47.699 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-14T21:35:47.717 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: set os type and version on server
2021-07-14T21:35:47.718 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-14T21:35:48.019 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64
2021-07-14T21:35:48.074 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="Red Hat Enterprise Linux"
2021-07-14T21:35:48.075 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="8.3 (Ootpa)"
2021-07-14T21:35:48.075 INFO:teuthology.orchestra.run.smithi059.stdout:ID="rhel"
2021-07-14T21:35:48.075 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="fedora"
2021-07-14T21:35:48.075 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="8.3"
2021-07-14T21:35:48.076 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el8"
2021-07-14T21:35:48.076 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-07-14T21:35:48.076 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31"
2021-07-14T21:35:48.076 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-07-14T21:35:48.077 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://www.redhat.com/"
2021-07-14T21:35:48.077 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-07-14T21:35:48.077 INFO:teuthology.orchestra.run.smithi059.stdout:
2021-07-14T21:35:48.077 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-07-14T21:35:48.078 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-07-14T21:35:48.078 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-07-14T21:35:48.078 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-07-14T21:35:48.080 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server
2021-07-14T21:35:48.095 INFO:teuthology.lock.ops:Updating [smithi103.front.sepia.ceph.com]: set os type and version on server
2021-07-14T21:35:48.095 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-14T21:35:48.375 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64
2021-07-14T21:35:48.429 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="Red Hat Enterprise Linux"
2021-07-14T21:35:48.430 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="8.3 (Ootpa)"
2021-07-14T21:35:48.430 INFO:teuthology.orchestra.run.smithi103.stdout:ID="rhel"
2021-07-14T21:35:48.431 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="fedora"
2021-07-14T21:35:48.431 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="8.3"
2021-07-14T21:35:48.432 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el8"
2021-07-14T21:35:48.432 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-07-14T21:35:48.433 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31"
2021-07-14T21:35:48.433 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-07-14T21:35:48.433 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://www.redhat.com/"
2021-07-14T21:35:48.433 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-07-14T21:35:48.433 INFO:teuthology.orchestra.run.smithi103.stdout:
2021-07-14T21:35:48.434 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-07-14T21:35:48.434 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-07-14T21:35:48.434 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-07-14T21:35:48.435 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-07-14T21:35:48.435 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server
2021-07-14T21:35:48.518 INFO:teuthology.dispatcher.supervisor:Running job 6271995
2021-07-14T21:35:48.519 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/virtualenv/bin/teuthology -v --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2021-07-14_21:27:35-rados:upgrade-wip-sage-testing-2021-07-14-1613-distro-basic-smithi/6271995 --name sage-2021-07-14_21:27:35-rados:upgrade-wip-sage-testing-2021-07-14-1613-distro-basic-smithi --description rados:upgrade/parallel/{0-distro$/{rhel_8.3_kubic_stable} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} -- /home/teuthworker/archive/sage-2021-07-14_21:27:35-rados:upgrade-wip-sage-testing-2021-07-14-1613-distro-basic-smithi/6271995/orig.config.yaml
2021-07-14T21:35:48.523 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sage-2021-07-14_21:27:35-rados:upgrade-wip-sage-testing-2021-07-14-1613-distro-basic-smithi/6271995
2021-07-14T21:35:48.524 INFO:teuthology.dispatcher.supervisor:Job PID: 8178
2021-07-14T21:35:48.524 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-07-15T09:36:57.615 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-07-15T09:36:58.561 INFO:teuthology.kill:Killing Pids: {8178}
2021-07-15T09:36:59.872 INFO:teuthology.kill:Nuking machines: ['smithi059', 'smithi103']
2021-07-15T09:37:00.838 INFO:teuthology.kill:2021-07-15 09:37:00,837.837 INFO:teuthology.nuke:targets:
2021-07-15T09:37:00.839 INFO:teuthology.kill:  smithi059.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4kPW1iUlmoPHlYZldludsMCF0udsu+hYoVMJuMumv2C9U/mEu7KyCc4YRBA93+I3ZIfIlub9dQmlTE+slH1QTjOPN0QZhzylz0fAfxdP+0qjFM2TtyWl2bfmhR/e9rEk/U2+C0S/Uoyz2Yd7EKzWA9LJFC2HJcKv/WDVWGAlWK8HfqAcInHzWd09Wsni/Rp818O2ODZi6pLeqGSkWzFQsUaBJK0XayKtG4X8mYJKBW9tOoRrt3waZkG6hG1gNEyu6N/I5XJPRPuP1sLEtAEz5gmaWvDvX6DJnyorWLgSmPnQNbdqE5qx/zLNLIls2OMAtGdtkYjiApM47WntZxSBNvsyFP4PLozF1kGMA2crwYUlnP8mk0RQ2PTc++a1b+c2Ea58GLl7lfLYFviAItWE7GYnWoO07SQS+Zb/Kh3c4mfD63Tmu6UvofSYQb/rxQCnWC6bv6hx84wrQJhxNgOV3kp7sL5WUphj3xMNbPJdYtN4mGPG1qKAM4WwnoNASJ/0=
2021-07-15T09:37:00.840 INFO:teuthology.kill:  smithi103.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwsawhKF0ict6X6fAk76BZNGYk/Xj5TvWqM2+CsLrFfFTNAQ45C3viF+TE8FPbPQrqPsfXruMxlyer5Bn9B1exRIfEWWxveUuQ50WKEHSY86xDw+mRJ4Cyw3FxeOng11mvrmaYziBA0RiSDgJbNDedrjp+ejDHgAqoURW8dXOsg3OE0nbZ7eMUSJ7PuzOm6S90n6cyNLFXwhqISY7GIfejh3uV9rkT62kDFXZmEtYjDkS7c8HeaWXR9MRICakgmDsi9yc4RA9TZdxNDdPlcDRnwNrdEkNgoIpPggf5nb3v8Shypn70zjULQnvAWZyGoKmjBkJj64bL6IN+deOywa/R71FutdWQPPrP1p1nqJEx2K7iDHtV1EbYNOKf5u0ueIIP0wcIThA3fB/mIhzjcJHbSN6QnW3wGDGf5e561FYbyQ15cEunKyXaV2d9lcRntisHqVaDzLGjTTtN5gOb+0wxzI1UJEFifZCXROg6XN145/aiYy3fGTZ1k2ptXVOdxlM=
2021-07-15T09:37:00.840 INFO:teuthology.kill:2021-07-15 09:37:00,838.838 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-15T09:37:00.841 INFO:teuthology.kill:2021-07-15 09:37:00,840.840 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-15T09:37:00.903 INFO:teuthology.kill:2021-07-15 09:37:00,902.902 INFO:teuthology.orchestra.console:Power off smithi103
2021-07-15T09:37:00.917 INFO:teuthology.kill:2021-07-15 09:37:00,916.916 INFO:teuthology.orchestra.console:Power off smithi059
2021-07-15T09:37:13.387 INFO:teuthology.kill:2021-07-15 09:37:13,387.387 INFO:teuthology.orchestra.console:Power off for smithi103 completed
2021-07-15T09:37:13.392 INFO:teuthology.kill:2021-07-15 09:37:13,392.392 INFO:teuthology.orchestra.console:Power off for smithi059 completed
2021-07-15T09:37:13.663 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a']
2021-07-15T09:37:13.665 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b']
2021-07-15T09:37:13.670 INFO:teuthology.misc:Compressing logs...
2021-07-15T09:37:13.682 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-15T09:37:32.120 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/teuthology/orchestra/remote.py", line 509, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_22d0fca188262bb2f460f46cb79ab21e54fb316f/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-07-15T09:37:32.211 INFO:teuthology.kill:No teuthology processes running
2021-07-15T09:37:33.297 INFO:teuthology.kill:Nuking machines: ['smithi059', 'smithi103']
2021-07-15T09:37:34.244 INFO:teuthology.kill:2021-07-15 09:37:34,243.243 INFO:teuthology.nuke:targets:
2021-07-15T09:37:34.245 INFO:teuthology.kill:  smithi059.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4kPW1iUlmoPHlYZldludsMCF0udsu+hYoVMJuMumv2C9U/mEu7KyCc4YRBA93+I3ZIfIlub9dQmlTE+slH1QTjOPN0QZhzylz0fAfxdP+0qjFM2TtyWl2bfmhR/e9rEk/U2+C0S/Uoyz2Yd7EKzWA9LJFC2HJcKv/WDVWGAlWK8HfqAcInHzWd09Wsni/Rp818O2ODZi6pLeqGSkWzFQsUaBJK0XayKtG4X8mYJKBW9tOoRrt3waZkG6hG1gNEyu6N/I5XJPRPuP1sLEtAEz5gmaWvDvX6DJnyorWLgSmPnQNbdqE5qx/zLNLIls2OMAtGdtkYjiApM47WntZxSBNvsyFP4PLozF1kGMA2crwYUlnP8mk0RQ2PTc++a1b+c2Ea58GLl7lfLYFviAItWE7GYnWoO07SQS+Zb/Kh3c4mfD63Tmu6UvofSYQb/rxQCnWC6bv6hx84wrQJhxNgOV3kp7sL5WUphj3xMNbPJdYtN4mGPG1qKAM4WwnoNASJ/0=
2021-07-15T09:37:34.245 INFO:teuthology.kill:  smithi103.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwsawhKF0ict6X6fAk76BZNGYk/Xj5TvWqM2+CsLrFfFTNAQ45C3viF+TE8FPbPQrqPsfXruMxlyer5Bn9B1exRIfEWWxveUuQ50WKEHSY86xDw+mRJ4Cyw3FxeOng11mvrmaYziBA0RiSDgJbNDedrjp+ejDHgAqoURW8dXOsg3OE0nbZ7eMUSJ7PuzOm6S90n6cyNLFXwhqISY7GIfejh3uV9rkT62kDFXZmEtYjDkS7c8HeaWXR9MRICakgmDsi9yc4RA9TZdxNDdPlcDRnwNrdEkNgoIpPggf5nb3v8Shypn70zjULQnvAWZyGoKmjBkJj64bL6IN+deOywa/R71FutdWQPPrP1p1nqJEx2K7iDHtV1EbYNOKf5u0ueIIP0wcIThA3fB/mIhzjcJHbSN6QnW3wGDGf5e561FYbyQ15cEunKyXaV2d9lcRntisHqVaDzLGjTTtN5gOb+0wxzI1UJEFifZCXROg6XN145/aiYy3fGTZ1k2ptXVOdxlM=
2021-07-15T09:37:34.264 INFO:teuthology.kill:2021-07-15 09:37:34,264.264 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-15T09:37:34.266 INFO:teuthology.kill:2021-07-15 09:37:34,266.266 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-15T09:37:34.319 INFO:teuthology.kill:2021-07-15 09:37:34,319.319 INFO:teuthology.orchestra.console:Power off smithi103
2021-07-15T09:37:34.331 INFO:teuthology.kill:2021-07-15 09:37:34,331.331 INFO:teuthology.orchestra.console:Power off smithi059
2021-07-15T09:37:34.458 INFO:teuthology.kill:2021-07-15 09:37:34,458.458 INFO:teuthology.orchestra.console:Power off for smithi103 completed
2021-07-15T09:37:34.469 INFO:teuthology.kill:2021-07-15 09:37:34,468.468 INFO:teuthology.orchestra.console:Power off for smithi059 completed
2021-07-15T09:37:34.596 INFO:teuthology.kill:2021-07-15 09:37:34,595.595 INFO:teuthology.lock.ops:unlocked smithi103.front.sepia.ceph.com
2021-07-15T09:37:34.607 INFO:teuthology.kill:2021-07-15 09:37:34,607.607 INFO:teuthology.lock.ops:unlocked smithi059.front.sepia.ceph.com
2021-07-15T09:39:34.849 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-07-15T09:39:34.964 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-07-15T09:39:34.969 INFO:teuthology.nuke:Checking targets against current locks
2021-07-15T09:39:35.328 INFO:teuthology.nuke:Not nuking smithi059.front.sepia.ceph.com because description doesn't match
2021-07-15T09:39:35.329 INFO:teuthology.nuke:Not nuking smithi103.front.sepia.ceph.com because description doesn't match