2021-05-29T06:16:44.976 INFO:root:teuthology version: 1.1.0-f359b10d
2021-05-29T06:16:45.094 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi016.front.sepia.ceph.com' reimaging
2021-05-29T06:16:45.095 INFO:teuthology.lock.ops:Updating [ubuntu@smithi016.front.sepia.ceph.com]: reset os type and version on server
2021-05-29T06:16:45.096 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server
2021-05-29T06:16:45.111 INFO:teuthology.lock.ops:Node 'ubuntu@smithi016.front.sepia.ceph.com' reimaging is complete
2021-05-29T06:16:45.111 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi074.front.sepia.ceph.com' reimaging
2021-05-29T06:16:45.111 INFO:teuthology.lock.ops:Updating [ubuntu@smithi074.front.sepia.ceph.com]: reset os type and version on server
2021-05-29T06:16:45.112 INFO:teuthology.lock.ops:Updating smithi074.front.sepia.ceph.com on lock server
2021-05-29T06:16:45.130 INFO:teuthology.lock.ops:Node 'ubuntu@smithi074.front.sepia.ceph.com' reimaging is complete
2021-05-29T06:16:45.309 INFO:teuthology.provision.fog.smithi016:Scheduling deploy of centos 8.3
2021-05-29T06:16:45.330 INFO:teuthology.provision.fog.smithi074:Scheduling deploy of centos 8.3
2021-05-29T06:16:45.653 INFO:teuthology.orchestra.console:Power off smithi074
2021-05-29T06:16:45.740 INFO:teuthology.orchestra.console:Power off smithi016
2021-05-29T06:16:54.027 INFO:teuthology.orchestra.console:Power off for smithi074 completed
2021-05-29T06:16:54.108 INFO:teuthology.orchestra.console:Power off for smithi016 completed
2021-05-29T06:16:54.129 INFO:teuthology.orchestra.console:Power on smithi074
2021-05-29T06:16:54.210 INFO:teuthology.orchestra.console:Power on smithi016
2021-05-29T06:16:58.387 INFO:teuthology.orchestra.console:Power on for smithi074 completed
2021-05-29T06:16:58.488 INFO:teuthology.provision.fog.smithi074:Waiting for deploy to finish
2021-05-29T06:17:02.694 INFO:teuthology.orchestra.console:Power on for smithi016 completed
2021-05-29T06:17:02.795 INFO:teuthology.provision.fog.smithi016:Waiting for deploy to finish
2021-05-29T06:21:16.463 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-05-29T06:21:16.796 INFO:teuthology.provision.fog.smithi016:Node is ready
2021-05-29T06:21:16.865 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016.front.sepia.ceph.com
2021-05-29T06:21:16.933 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16 smithi016.front.sepia.ceph.com smithi016
2021-05-29T06:21:17.341 INFO:teuthology.provision.fog.smithi016:Deploy complete!
2021-05-29T06:21:39.270 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-05-29T06:21:39.637 INFO:teuthology.provision.fog.smithi074:Node is ready
2021-05-29T06:21:39.651 INFO:teuthology.orchestra.run.smithi074.stdout:smithi074.front.sepia.ceph.com
2021-05-29T06:21:39.706 INFO:teuthology.orchestra.run.smithi074.stdout:172.21.15.74 smithi074.front.sepia.ceph.com smithi074
2021-05-29T06:21:40.029 INFO:teuthology.provision.fog.smithi074:Deploy complete!
2021-05-29T06:21:40.183 INFO:teuthology.lock.ops:Checking smithi016.front.sepia.ceph.com
2021-05-29T06:21:40.184 INFO:teuthology.lock.ops:New key found. Updating...
2021-05-29T06:21:40.201 INFO:teuthology.lock.ops:Checking smithi074.front.sepia.ceph.com
2021-05-29T06:21:40.201 INFO:teuthology.lock.ops:New key found. Updating...
2021-05-29T06:21:40.216 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: set os type and version on server
2021-05-29T06:21:40.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-05-29T06:21:40.521 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64
2021-05-29T06:21:40.577 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Linux"
2021-05-29T06:21:40.578 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8"
2021-05-29T06:21:40.578 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos"
2021-05-29T06:21:40.578 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora"
2021-05-29T06:21:40.578 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8"
2021-05-29T06:21:40.579 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8"
2021-05-29T06:21:40.579 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Linux 8"
2021-05-29T06:21:40.579 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31"
2021-05-29T06:21:40.580 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-05-29T06:21:40.580 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/"
2021-05-29T06:21:40.580 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-05-29T06:21:40.580 INFO:teuthology.orchestra.run.smithi016.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-05-29T06:21:40.581 INFO:teuthology.orchestra.run.smithi016.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-05-29T06:21:40.582 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server
2021-05-29T06:21:40.598 INFO:teuthology.lock.ops:Updating [smithi074.front.sepia.ceph.com]: set os type and version on server
2021-05-29T06:21:40.599 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-05-29T06:21:40.883 INFO:teuthology.orchestra.run.smithi074.stdout:x86_64
2021-05-29T06:21:40.938 INFO:teuthology.orchestra.run.smithi074.stdout:NAME="CentOS Linux"
2021-05-29T06:21:40.938 INFO:teuthology.orchestra.run.smithi074.stdout:VERSION="8"
2021-05-29T06:21:40.939 INFO:teuthology.orchestra.run.smithi074.stdout:ID="centos"
2021-05-29T06:21:40.939 INFO:teuthology.orchestra.run.smithi074.stdout:ID_LIKE="rhel fedora"
2021-05-29T06:21:40.939 INFO:teuthology.orchestra.run.smithi074.stdout:VERSION_ID="8"
2021-05-29T06:21:40.939 INFO:teuthology.orchestra.run.smithi074.stdout:PLATFORM_ID="platform:el8"
2021-05-29T06:21:40.940 INFO:teuthology.orchestra.run.smithi074.stdout:PRETTY_NAME="CentOS Linux 8"
2021-05-29T06:21:40.940 INFO:teuthology.orchestra.run.smithi074.stdout:ANSI_COLOR="0;31"
2021-05-29T06:21:40.940 INFO:teuthology.orchestra.run.smithi074.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-05-29T06:21:40.940 INFO:teuthology.orchestra.run.smithi074.stdout:HOME_URL="https://centos.org/"
2021-05-29T06:21:40.941 INFO:teuthology.orchestra.run.smithi074.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-05-29T06:21:40.941 INFO:teuthology.orchestra.run.smithi074.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-05-29T06:21:40.941 INFO:teuthology.orchestra.run.smithi074.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-05-29T06:21:40.942 INFO:teuthology.lock.ops:Updating smithi074.front.sepia.ceph.com on lock server
2021-05-29T06:21:41.014 INFO:teuthology.dispatcher.supervisor:Running job 6141280
2021-05-29T06:21:41.015 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology -v --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2021-05-29_04:39:10-rados-wip-sage3-testing-2021-05-28-2225-distro-basic-smithi/6141280 --name sage-2021-05-29_04:39:10-rados-wip-sage3-testing-2021-05-28-2225-distro-basic-smithi --description rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/snaps-few-objects} -- /home/teuthworker/archive/sage-2021-05-29_04:39:10-rados-wip-sage3-testing-2021-05-28-2225-distro-basic-smithi/6141280/orig.config.yaml
2021-05-29T06:21:41.019 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sage-2021-05-29_04:39:10-rados-wip-sage3-testing-2021-05-28-2225-distro-basic-smithi/6141280
2021-05-29T06:21:41.020 INFO:teuthology.dispatcher.supervisor:Job PID: 9523
2021-05-29T06:21:41.021 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-05-29T18:22:25.577 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-05-29T18:22:25.774 INFO:teuthology.kill:Killing Pids: {9523}
2021-05-29T18:22:26.891 INFO:teuthology.kill:Nuking machines: ['smithi016', 'smithi074']
2021-05-29T18:22:27.836 INFO:teuthology.kill:2021-05-29 18:22:27,835.835 INFO:teuthology.nuke:targets:
2021-05-29T18:22:27.836 INFO:teuthology.kill:  smithi016.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzhU9DsU8UzX9xNVDnMK14S0W+33Jjjcp/lR4gRMz2Uy1kZS522t985cXprPxEASg1o37sUc+8zxwEoiqEGYebuH3j5OWx9luD6ezsG2fa/qdEfmafve8+1x0X3N4eLS6cA4fEq+ESMn7X2tLR9TeK7Y83zI8MsTFeLfovB4hA2yguwAqA6+5Nmp4109jtlnShGz/CZWRj949Z4bZAOc93VnUcDwmkvNqLoj8gi+AqfIAYWqivYhVlBpg2qiySXmjaOM7M86dKqvKskxaCPcZVtg5X/+pPDDPBTR9rY9UbqK64TIS0RO7N/o3wwh/MAd9VrVa2gBKLrQQVRWdez2PCE/7qtsFSEKugYILOMysOTVh0NqK2OhbEWBC4UJXrE+WYAisBy08lv8Sxl+GSqtqgWnPJFkQqeFzlw8vVNnRxdSXaoIS/4hlhWxNJ1cB+ETCSIkZot8jdQ9CECauzvZq4BpHHU3hI7RlLtNmSFg/+tW89MB7p3oh8KBkjTSeCyN0=
2021-05-29T18:22:27.837 INFO:teuthology.kill:  smithi074.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYpFT9rWGAN4e+HSw7HVfMlYv8YU3JF2CaH0pkFFinsgpFOkuiY+mlVXHWtHUwIfawPB5SZbOwLiwpJ9umhSXueEI/T6h78XqViVD+XZJwU+1Wr26BQs/Ql3oVtsnHskzDey6T6OnYbOOjrP0ON35bbY8EBZ5k5j2qiBgiY/tk8DW9cXPVp3tAWdFDL3MijMp7RT/cFivWQg2nocyT9zBkhvGPuvLiqEagySJiIt8K8HT7vzguMrddtsDH3d9xvyuxsc3Og2KOYUd5+uYchGhtdlo+mHyVNa0T70o+N1GNN5n/6ViMwt3v4WwH7wDhs3a4D/vFEA4jcHscQLmkf/wZRyvo/+6dsT6108vTzNqD3eEceJH9IWQsgz4xEMXFFwnINaY5mt/HnV5QX3KoQtRp9MkFeWz3MPyrOU41ZfCZeTkyN7PrFThJtg/PKGziQjAGus9V3VfTHSIs+DdytUIRZbcOBuBhb/u8hRrCZhtv+pCDNTeFD5zE9fUGRGZoOnc=
2021-05-29T18:22:27.837 INFO:teuthology.kill:2021-05-29 18:22:27,836.836 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-29T18:22:27.838 INFO:teuthology.kill:2021-05-29 18:22:27,838.838 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-29T18:22:27.897 INFO:teuthology.kill:2021-05-29 18:22:27,897.897 INFO:teuthology.orchestra.console:Power off smithi074
2021-05-29T18:22:27.912 INFO:teuthology.kill:2021-05-29 18:22:27,912.912 INFO:teuthology.orchestra.console:Power off smithi016
2021-05-29T18:22:36.275 INFO:teuthology.kill:2021-05-29 18:22:36,275.275 INFO:teuthology.orchestra.console:Power off for smithi016 completed
2021-05-29T18:22:40.377 INFO:teuthology.kill:2021-05-29 18:22:40,377.377 INFO:teuthology.orchestra.console:Power off for smithi074 completed
2021-05-29T18:22:40.638 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2']
2021-05-29T18:22:40.639 INFO:teuthology.task.internal:roles: ubuntu@smithi074.front.sepia.ceph.com - ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0']
2021-05-29T18:22:40.639 INFO:teuthology.misc:Compressing logs...
2021-05-29T18:22:40.640 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-05-29T18:23:15.151 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/remote.py", line 509, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-05-29T18:23:15.207 INFO:teuthology.kill:No teuthology processes running
2021-05-29T18:23:16.283 INFO:teuthology.kill:Nuking machines: ['smithi016', 'smithi074']
2021-05-29T18:23:17.231 INFO:teuthology.kill:2021-05-29 18:23:17,231.231 INFO:teuthology.nuke:targets:
2021-05-29T18:23:17.232 INFO:teuthology.kill:  smithi016.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzhU9DsU8UzX9xNVDnMK14S0W+33Jjjcp/lR4gRMz2Uy1kZS522t985cXprPxEASg1o37sUc+8zxwEoiqEGYebuH3j5OWx9luD6ezsG2fa/qdEfmafve8+1x0X3N4eLS6cA4fEq+ESMn7X2tLR9TeK7Y83zI8MsTFeLfovB4hA2yguwAqA6+5Nmp4109jtlnShGz/CZWRj949Z4bZAOc93VnUcDwmkvNqLoj8gi+AqfIAYWqivYhVlBpg2qiySXmjaOM7M86dKqvKskxaCPcZVtg5X/+pPDDPBTR9rY9UbqK64TIS0RO7N/o3wwh/MAd9VrVa2gBKLrQQVRWdez2PCE/7qtsFSEKugYILOMysOTVh0NqK2OhbEWBC4UJXrE+WYAisBy08lv8Sxl+GSqtqgWnPJFkQqeFzlw8vVNnRxdSXaoIS/4hlhWxNJ1cB+ETCSIkZot8jdQ9CECauzvZq4BpHHU3hI7RlLtNmSFg/+tW89MB7p3oh8KBkjTSeCyN0=
2021-05-29T18:23:17.233 INFO:teuthology.kill:  smithi074.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYpFT9rWGAN4e+HSw7HVfMlYv8YU3JF2CaH0pkFFinsgpFOkuiY+mlVXHWtHUwIfawPB5SZbOwLiwpJ9umhSXueEI/T6h78XqViVD+XZJwU+1Wr26BQs/Ql3oVtsnHskzDey6T6OnYbOOjrP0ON35bbY8EBZ5k5j2qiBgiY/tk8DW9cXPVp3tAWdFDL3MijMp7RT/cFivWQg2nocyT9zBkhvGPuvLiqEagySJiIt8K8HT7vzguMrddtsDH3d9xvyuxsc3Og2KOYUd5+uYchGhtdlo+mHyVNa0T70o+N1GNN5n/6ViMwt3v4WwH7wDhs3a4D/vFEA4jcHscQLmkf/wZRyvo/+6dsT6108vTzNqD3eEceJH9IWQsgz4xEMXFFwnINaY5mt/HnV5QX3KoQtRp9MkFeWz3MPyrOU41ZfCZeTkyN7PrFThJtg/PKGziQjAGus9V3VfTHSIs+DdytUIRZbcOBuBhb/u8hRrCZhtv+pCDNTeFD5zE9fUGRGZoOnc=
2021-05-29T18:23:17.252 INFO:teuthology.kill:2021-05-29 18:23:17,251.251 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-29T18:23:17.254 INFO:teuthology.kill:2021-05-29 18:23:17,254.254 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-29T18:23:17.314 INFO:teuthology.kill:2021-05-29 18:23:17,314.314 INFO:teuthology.orchestra.console:Power off smithi074
2021-05-29T18:23:17.327 INFO:teuthology.kill:2021-05-29 18:23:17,327.327 INFO:teuthology.orchestra.console:Power off smithi016
2021-05-29T18:23:17.457 INFO:teuthology.kill:2021-05-29 18:23:17,456.456 INFO:teuthology.orchestra.console:Power off for smithi074 completed
2021-05-29T18:23:17.465 INFO:teuthology.kill:2021-05-29 18:23:17,465.465 INFO:teuthology.orchestra.console:Power off for smithi016 completed
2021-05-29T18:23:17.602 INFO:teuthology.kill:2021-05-29 18:23:17,601.601 INFO:teuthology.lock.ops:unlocked smithi074.front.sepia.ceph.com
2021-05-29T18:23:17.605 INFO:teuthology.kill:2021-05-29 18:23:17,605.605 INFO:teuthology.lock.ops:unlocked smithi016.front.sepia.ceph.com
2021-05-29T18:25:17.886 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-05-29T18:25:17.982 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-05-29T18:25:17.982 INFO:teuthology.nuke:Checking targets against current locks
2021-05-29T18:25:18.072 INFO:teuthology.nuke:Not nuking smithi074.front.sepia.ceph.com because description doesn't match
2021-05-29T18:25:18.072 INFO:teuthology.nuke:Not nuking smithi016.front.sepia.ceph.com because description doesn't match