2021-07-24T08:44:00.410 INFO:root:teuthology version: 1.1.0-b6f3efab
2021-07-24T08:44:00.509 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi032.front.sepia.ceph.com' reimaging
2021-07-24T08:44:00.510 INFO:teuthology.lock.ops:Updating [ubuntu@smithi032.front.sepia.ceph.com]: reset os type and version on server
2021-07-24T08:44:00.510 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server
2021-07-24T08:44:00.525 INFO:teuthology.lock.ops:Node 'ubuntu@smithi032.front.sepia.ceph.com' reimaging is complete
2021-07-24T08:44:00.526 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi181.front.sepia.ceph.com' reimaging
2021-07-24T08:44:00.526 INFO:teuthology.lock.ops:Updating [ubuntu@smithi181.front.sepia.ceph.com]: reset os type and version on server
2021-07-24T08:44:00.526 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server
2021-07-24T08:44:00.544 INFO:teuthology.lock.ops:Node 'ubuntu@smithi181.front.sepia.ceph.com' reimaging is complete
2021-07-24T08:44:00.773 INFO:teuthology.provision.fog.smithi032:Scheduling deploy of rhel 8.3
2021-07-24T08:44:00.778 INFO:teuthology.provision.fog.smithi181:Scheduling deploy of rhel 8.3
2021-07-24T08:44:01.136 INFO:teuthology.orchestra.console:Power off smithi032
2021-07-24T08:44:01.150 INFO:teuthology.orchestra.console:Power off smithi181
2021-07-24T08:44:01.279 INFO:teuthology.orchestra.console:Power off for smithi032 completed
2021-07-24T08:44:01.290 INFO:teuthology.orchestra.console:Power off for smithi181 completed
2021-07-24T08:44:01.381 INFO:teuthology.orchestra.console:Power on smithi032
2021-07-24T08:44:01.392 INFO:teuthology.orchestra.console:Power on smithi181
2021-07-24T08:44:05.637 INFO:teuthology.orchestra.console:Power on for smithi032 completed
2021-07-24T08:44:05.642 INFO:teuthology.orchestra.console:Power on for smithi181 completed
2021-07-24T08:44:05.739 INFO:teuthology.provision.fog.smithi032:Waiting for deploy to finish
2021-07-24T08:44:05.743 INFO:teuthology.provision.fog.smithi181:Waiting for deploy to finish
2021-07-24T08:48:28.836 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-24T08:48:29.166 INFO:teuthology.provision.fog.smithi032:Node is ready
2021-07-24T08:48:29.182 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032.front.sepia.ceph.com
2021-07-24T08:48:29.237 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32 smithi032.front.sepia.ceph.com smithi032
2021-07-24T08:48:29.580 INFO:teuthology.provision.fog.smithi032:Deploy complete!
2021-07-24T08:48:37.838 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-24T08:48:38.159 INFO:teuthology.provision.fog.smithi181:Node is ready
2021-07-24T08:48:38.173 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181.front.sepia.ceph.com
2021-07-24T08:48:38.228 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181 smithi181.front.sepia.ceph.com smithi181
2021-07-24T08:48:38.544 INFO:teuthology.provision.fog.smithi181:Deploy complete!
2021-07-24T08:48:38.667 INFO:teuthology.lock.ops:Checking smithi032.front.sepia.ceph.com
2021-07-24T08:48:38.668 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-24T08:48:38.684 INFO:teuthology.lock.ops:Checking smithi181.front.sepia.ceph.com
2021-07-24T08:48:38.685 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-24T08:48:38.699 INFO:teuthology.lock.ops:Updating [smithi032.front.sepia.ceph.com]: set os type and version on server
2021-07-24T08:48:38.700 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-24T08:48:38.995 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64
2021-07-24T08:48:39.050 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Red Hat Enterprise Linux"
2021-07-24T08:48:39.051 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="8.3 (Ootpa)"
2021-07-24T08:48:39.051 INFO:teuthology.orchestra.run.smithi032.stdout:ID="rhel"
2021-07-24T08:48:39.051 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="fedora"
2021-07-24T08:48:39.051 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="8.3"
2021-07-24T08:48:39.051 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el8"
2021-07-24T08:48:39.052 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-07-24T08:48:39.052 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31"
2021-07-24T08:48:39.052 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-07-24T08:48:39.052 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.redhat.com/"
2021-07-24T08:48:39.053 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-07-24T08:48:39.053 INFO:teuthology.orchestra.run.smithi032.stdout:
2021-07-24T08:48:39.053 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-07-24T08:48:39.053 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-07-24T08:48:39.054 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-07-24T08:48:39.054 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-07-24T08:48:39.055 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server
2021-07-24T08:48:39.077 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: set os type and version on server
2021-07-24T08:48:39.078 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-24T08:48:39.354 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64
2021-07-24T08:48:39.408 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Red Hat Enterprise Linux"
2021-07-24T08:48:39.408 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8.3 (Ootpa)"
2021-07-24T08:48:39.408 INFO:teuthology.orchestra.run.smithi181.stdout:ID="rhel"
2021-07-24T08:48:39.408 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="fedora"
2021-07-24T08:48:39.408 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8.3"
2021-07-24T08:48:39.409 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8"
2021-07-24T08:48:39.409 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-07-24T08:48:39.409 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31"
2021-07-24T08:48:39.409 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-07-24T08:48:39.410 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.redhat.com/"
2021-07-24T08:48:39.410 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-07-24T08:48:39.410 INFO:teuthology.orchestra.run.smithi181.stdout:
2021-07-24T08:48:39.410 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-07-24T08:48:39.410 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-07-24T08:48:39.411 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-07-24T08:48:39.411 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-07-24T08:48:39.411 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server
2021-07-24T08:48:39.502 INFO:teuthology.dispatcher.supervisor:Running job 6290005
2021-07-24T08:48:39.503 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/virtualenv/bin/teuthology -v --owner scheduled_kchai@teuthology --archive /home/teuthworker/archive/kchai-2021-07-24_05:56:34-upgrade-wip-kefu-testing-2021-07-24-1158-distro-basic-smithi/6290005 --name kchai-2021-07-24_05:56:34-upgrade-wip-kefu-testing-2021-07-24-1158-distro-basic-smithi --description upgrade/octopus-x/stress-split/{0-distro/rhel_8.3_kubic_stable 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} -- /home/teuthworker/archive/kchai-2021-07-24_05:56:34-upgrade-wip-kefu-testing-2021-07-24-1158-distro-basic-smithi/6290005/orig.config.yaml
2021-07-24T08:48:39.506 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/kchai-2021-07-24_05:56:34-upgrade-wip-kefu-testing-2021-07-24-1158-distro-basic-smithi/6290005
2021-07-24T08:48:39.507 INFO:teuthology.dispatcher.supervisor:Job PID: 26258
2021-07-24T08:48:39.507 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-07-24T20:49:27.768 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-07-24T20:49:28.075 INFO:teuthology.kill:Killing Pids: {26258}
2021-07-24T20:49:29.421 INFO:teuthology.kill:Nuking machines: ['smithi032', 'smithi181']
2021-07-24T20:49:30.436 INFO:teuthology.kill:2021-07-24 20:49:30,436.436 INFO:teuthology.nuke:targets:
2021-07-24T20:49:30.437 INFO:teuthology.kill:  smithi032.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjKoobl1mvVF0rLxtVv3CqXaVSXyTFDunSBPfAzS0NgqKjqhsKCDc5LGKt6fyNN0vy/gr6xtVIayJ+GM1CGmiPBIGcT+4g+FpAdSt8qq6FFNqYfHkQig7omELPssoENrnPKKfUtE+wbbn4RfTdGoL2YsH5CtYcSlt4qWjnIPcF3uTrf38aPIuzXYZ6o6MALqDRTAqOFY22fnU/Fc7/1k/bxszAP+wzywbtPFzOZBMBxDutHjh78A0bTR9AqMQFIEt+B0H3FvCT9O22MAMdv+v6d20uf9sHu4+dpIpe/Q4oyYjJILAkhJT26n9GjDp4pbj3dRUSzWffJEoLE5Or5DECBhy3Up2+PyzZnylEQ0F7BZh9g5ET3gR0DgvdzxHenovx93lN41Zn2eUPrzZumBmc3ydiOJeKp8SyFxTjwh3xjGk1blNBXlzXxSzEGzKUZQp6jJgzshOs+1RWzA/bJJbMRaxUJ/OPapiqryh8L53uLBwkT58DNF9YwrZzA1p5J78=
2021-07-24T20:49:30.438 INFO:teuthology.kill:  smithi181.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuZud9yAkQHU5TymUJDS7cmSER+kqwhJ+GryHs3sosKBbAgLv59cBJ2NoJqlckAAPi1p7nnld3c4KMKYJP3rPSOpz9XaRp/5jlhb9PrNCK4IKtZ6eU32nn345gmUktA/+IcXiUfGP1VRv2cbNRCuhwCASnHCHkFIStnZ1/QFR+szubC5V1ztn98fn2/TyD8lrMBls95avGJuBLxjeQ2wxz6URH1Yts6VHUKzTe3v5HkYHhbLx+86Cl2hvP5ktlx3+XuCh/O95eTT/LaJGfgbwuMX6lv7SDKg3moEA2l+QsCSLrjRDgW4NI9CWABHo0OsUcazI8h+1TUUGMu4aTJjsOwS4Zskin/vn+h/OjOU4Ao8p4Ofa9eJlXE4KkX5O56tpLgC/+opzGrHRymixSjqMZZzkGsNBd3Surd+c6sHEkSDFHjOENLv60PTThzGlTHpCNsfyvB96rZvo6Q12CZHm0CDyua7Lw5jJvhRoni6WA0V0O3Dc3whoLHmXPfsw/FWk=
2021-07-24T20:49:30.438 INFO:teuthology.kill:2021-07-24 20:49:30,436.436 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-24T20:49:30.439 INFO:teuthology.kill:2021-07-24 20:49:30,439.439 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-24T20:49:30.501 INFO:teuthology.kill:2021-07-24 20:49:30,501.501 INFO:teuthology.orchestra.console:Power off smithi032
2021-07-24T20:49:30.529 INFO:teuthology.kill:2021-07-24 20:49:30,528.528 INFO:teuthology.orchestra.console:Power off smithi181
2021-07-24T20:49:38.900 INFO:teuthology.kill:2021-07-24 20:49:38,899.899 INFO:teuthology.orchestra.console:Power off for smithi181 completed
2021-07-24T20:49:43.002 INFO:teuthology.kill:2021-07-24 20:49:43,001.001 INFO:teuthology.orchestra.console:Power off for smithi032 completed
2021-07-24T20:49:43.325 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a']
2021-07-24T20:49:43.326 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b']
2021-07-24T20:49:43.326 INFO:teuthology.misc:Compressing logs...
2021-07-24T20:49:43.327 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-24T20:50:43.387 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/remote.py", line 509, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-07-24T20:50:43.534 INFO:teuthology.kill:No teuthology processes running
2021-07-24T20:50:45.023 INFO:teuthology.kill:Nuking machines: ['smithi032', 'smithi181']
2021-07-24T20:50:46.268 INFO:teuthology.kill:2021-07-24 20:50:46,265.265 INFO:teuthology.nuke:targets:
2021-07-24T20:50:46.269 INFO:teuthology.kill:  smithi032.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjKoobl1mvVF0rLxtVv3CqXaVSXyTFDunSBPfAzS0NgqKjqhsKCDc5LGKt6fyNN0vy/gr6xtVIayJ+GM1CGmiPBIGcT+4g+FpAdSt8qq6FFNqYfHkQig7omELPssoENrnPKKfUtE+wbbn4RfTdGoL2YsH5CtYcSlt4qWjnIPcF3uTrf38aPIuzXYZ6o6MALqDRTAqOFY22fnU/Fc7/1k/bxszAP+wzywbtPFzOZBMBxDutHjh78A0bTR9AqMQFIEt+B0H3FvCT9O22MAMdv+v6d20uf9sHu4+dpIpe/Q4oyYjJILAkhJT26n9GjDp4pbj3dRUSzWffJEoLE5Or5DECBhy3Up2+PyzZnylEQ0F7BZh9g5ET3gR0DgvdzxHenovx93lN41Zn2eUPrzZumBmc3ydiOJeKp8SyFxTjwh3xjGk1blNBXlzXxSzEGzKUZQp6jJgzshOs+1RWzA/bJJbMRaxUJ/OPapiqryh8L53uLBwkT58DNF9YwrZzA1p5J78=
2021-07-24T20:50:46.270 INFO:teuthology.kill:  smithi181.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuZud9yAkQHU5TymUJDS7cmSER+kqwhJ+GryHs3sosKBbAgLv59cBJ2NoJqlckAAPi1p7nnld3c4KMKYJP3rPSOpz9XaRp/5jlhb9PrNCK4IKtZ6eU32nn345gmUktA/+IcXiUfGP1VRv2cbNRCuhwCASnHCHkFIStnZ1/QFR+szubC5V1ztn98fn2/TyD8lrMBls95avGJuBLxjeQ2wxz6URH1Yts6VHUKzTe3v5HkYHhbLx+86Cl2hvP5ktlx3+XuCh/O95eTT/LaJGfgbwuMX6lv7SDKg3moEA2l+QsCSLrjRDgW4NI9CWABHo0OsUcazI8h+1TUUGMu4aTJjsOwS4Zskin/vn+h/OjOU4Ao8p4Ofa9eJlXE4KkX5O56tpLgC/+opzGrHRymixSjqMZZzkGsNBd3Surd+c6sHEkSDFHjOENLv60PTThzGlTHpCNsfyvB96rZvo6Q12CZHm0CDyua7Lw5jJvhRoni6WA0V0O3Dc3whoLHmXPfsw/FWk=
2021-07-24T20:50:46.294 INFO:teuthology.kill:2021-07-24 20:50:46,294.294 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-24T20:50:46.308 INFO:teuthology.kill:2021-07-24 20:50:46,307.307 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-24T20:50:46.394 INFO:teuthology.kill:2021-07-24 20:50:46,393.393 INFO:teuthology.orchestra.console:Power off smithi032
2021-07-24T20:50:46.456 INFO:teuthology.kill:2021-07-24 20:50:46,455.455 INFO:teuthology.orchestra.console:Power off smithi181
2021-07-24T20:50:46.604 INFO:teuthology.kill:2021-07-24 20:50:46,603.603 INFO:teuthology.orchestra.console:Power off for smithi032 completed
2021-07-24T20:50:46.621 INFO:teuthology.kill:2021-07-24 20:50:46,621.621 INFO:teuthology.orchestra.console:Power off for smithi181 completed
2021-07-24T20:50:46.751 INFO:teuthology.kill:2021-07-24 20:50:46,750.750 INFO:teuthology.lock.ops:unlocked smithi032.front.sepia.ceph.com
2021-07-24T20:50:46.765 INFO:teuthology.kill:2021-07-24 20:50:46,765.765 INFO:teuthology.lock.ops:unlocked smithi181.front.sepia.ceph.com
2021-07-24T20:52:47.091 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-07-24T20:52:47.240 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-07-24T20:52:47.240 INFO:teuthology.nuke:Checking targets against current locks
2021-07-24T20:52:47.325 INFO:teuthology.nuke:Not nuking smithi032.front.sepia.ceph.com because description doesn't match
2021-07-24T20:52:47.325 INFO:teuthology.nuke:Not nuking smithi181.front.sepia.ceph.com because description doesn't match