2021-03-12T01:08:04.446 INFO:root:teuthology version: 1.1.0-b7dbd64f 2021-03-12T01:08:04.567 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi092.front.sepia.ceph.com' reimaging 2021-03-12T01:08:04.568 INFO:teuthology.lock.ops:Updating [ubuntu@smithi092.front.sepia.ceph.com]: reset os type and version on server 2021-03-12T01:08:04.568 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2021-03-12T01:08:04.587 INFO:teuthology.lock.ops:Node 'ubuntu@smithi092.front.sepia.ceph.com' reimaging is complete 2021-03-12T01:08:04.588 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging 2021-03-12T01:08:04.588 INFO:teuthology.lock.ops:Updating [ubuntu@smithi136.front.sepia.ceph.com]: reset os type and version on server 2021-03-12T01:08:04.588 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2021-03-12T01:08:04.604 INFO:teuthology.lock.ops:Node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging is complete 2021-03-12T01:08:04.806 INFO:teuthology.provision.fog.smithi092:Scheduling deploy of centos 8.1 2021-03-12T01:08:04.821 INFO:teuthology.provision.fog.smithi136:Scheduling deploy of centos 8.1 2021-03-12T01:08:05.227 INFO:teuthology.orchestra.console:Power off smithi092 2021-03-12T01:08:05.295 INFO:teuthology.orchestra.console:Power off smithi136 2021-03-12T01:08:13.660 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2021-03-12T01:08:13.761 INFO:teuthology.orchestra.console:Power on smithi136 2021-03-12T01:08:17.720 INFO:teuthology.orchestra.console:Power off for smithi092 completed 2021-03-12T01:08:17.822 INFO:teuthology.orchestra.console:Power on smithi092 2021-03-12T01:08:22.225 INFO:teuthology.orchestra.console:Power on for smithi136 completed 2021-03-12T01:08:22.326 INFO:teuthology.provision.fog.smithi136:Waiting for deploy to finish 2021-03-12T01:08:26.192 INFO:teuthology.orchestra.console:Power on for smithi092 completed 2021-03-12T01:08:26.293 INFO:teuthology.provision.fog.smithi092:Waiting for deploy to finish 2021-03-12T01:12:58.614 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-12T01:12:58.970 INFO:teuthology.provision.fog.smithi092:Node is ready 2021-03-12T01:12:58.986 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092.front.sepia.ceph.com 2021-03-12T01:12:59.041 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92 smithi092.front.sepia.ceph.com smithi092 2021-03-12T01:12:59.385 INFO:teuthology.provision.fog.smithi092:Deploy complete! 2021-03-12T01:13:03.820 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-12T01:13:04.154 INFO:teuthology.provision.fog.smithi136:Node is ready 2021-03-12T01:13:04.169 INFO:teuthology.orchestra.run.smithi136.stdout:smithi136.front.sepia.ceph.com 2021-03-12T01:13:04.224 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.15.136 smithi136.front.sepia.ceph.com smithi136 2021-03-12T01:13:04.559 INFO:teuthology.provision.fog.smithi136:Deploy complete! 2021-03-12T01:13:04.707 INFO:teuthology.lock.ops:Checking smithi092.front.sepia.ceph.com 2021-03-12T01:13:04.708 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-12T01:13:04.724 INFO:teuthology.lock.ops:Checking smithi136.front.sepia.ceph.com 2021-03-12T01:13:04.725 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-12T01:13:04.740 INFO:teuthology.lock.ops:Updating [smithi092.front.sepia.ceph.com]: set os type and version on server 2021-03-12T01:13:04.740 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-12T01:13:05.050 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2021-03-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Linux" 2021-03-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="8 (Core)" 2021-03-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2021-03-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2021-03-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="8" 2021-03-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el8" 2021-03-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-03-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2021-03-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-03-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://www.centos.org/" 2021-03-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-03-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi092.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-03-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi092.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-03-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-03-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-03-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-12T01:13:05.111 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2021-03-12T01:13:05.179 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: set os type and version on server 2021-03-12T01:13:05.179 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-12T01:13:05.475 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2021-03-12T01:13:05.528 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Linux" 2021-03-12T01:13:05.528 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="8 (Core)" 2021-03-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2021-03-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2021-03-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="8" 2021-03-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el8" 2021-03-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-03-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2021-03-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-03-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.centos.org/" 2021-03-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-03-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-03-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi136.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-03-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi136.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-03-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-03-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-03-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: 2021-03-12T01:13:05.532 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2021-03-12T01:13:05.677 INFO:teuthology.dispatcher.supervisor:Running job 5957256 2021-03-12T01:13:05.677 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-03-12_01:01:15-rados-octopus-distro-basic-smithi/5957256 --name yuriw-2021-03-12_01:01:15-rados-octopus-distro-basic-smithi --description rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} -- /home/teuthworker/archive/yuriw-2021-03-12_01:01:15-rados-octopus-distro-basic-smithi/5957256/orig.config.yaml 2021-03-12T01:13:05.682 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-03-12_01:01:15-rados-octopus-distro-basic-smithi/5957256 2021-03-12T01:13:05.683 INFO:teuthology.dispatcher.supervisor:Job PID: 14916 2021-03-12T01:13:05.683 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-03-12T13:13:56.827 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-03-12T13:13:57.057 INFO:teuthology.kill:Killing Pids: {14916} 2021-03-12T13:13:59.115 INFO:teuthology.kill:Nuking machines: ['smithi092', 'smithi136'] 2021-03-12T13:14:01.068 INFO:teuthology.kill:2021-03-12 13:14:01,068.068 INFO:teuthology.nuke:targets: 2021-03-12T13:14:01.073 INFO:teuthology.kill: smithi092.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDyF/RqNcXWcmkiDt+H/IzfM50DNu9ov78V8Z0wuP8Qwny1y1SqpaY+rmZV1cLFz1ItXCs+JYA/7RwKrQiKdpzL4acLY9MyYMagomc6Kv3ishoVh0AwJJBFPvSJ7OlNsHIQF/RV/WpKjiYNOonHhvPbcTMSTfQRCTDDdKd6AuiwdYbzKksmNXdmNYmJ3hqu5Cx1fmUXwU5DQzcJe4mpCigMYTnTy0bqiSzHYCkwyFKWvIiLG0zLdkZF7yaIgz/WSc3PuyXMfZxd9M4OTOSCKD8o2BopUHxnFox4auMZqIihrw1M9tYgrAzmJxI5muaK/XZMDw8OP2R8/tz7D+xUwWuRunQfHNOiBOm/wLwrmaEfiWv6+Uh4dBCNEbvlXft6M2JZUnh1gSRnOToLa9XYbNkGKJ+d4rgH0nYKG/ob9F32q/2GV4Qao8lxccODvc5vOjcraNXSpvoYR9c1t4zcfULLqvn5ggUzfIOycx/WE9lfD/Vag0dQQ8gWlXo4WJ4YD+M= 2021-03-12T13:14:01.073 INFO:teuthology.kill: smithi136.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTOrLsihQJ8eJWuImubvRAl25RXYDPgDSJKzwIGaBPsJjz4p+etAOEF42j6PE8QBLBdCyHFCZ82eW78l5tSBuV8x2kRb9tdrZFwcPF/7K0RK0js3+ci7Ej6svFt2qQb2A9QqctWbdd+C8m0JpqXtGzfUQykSQRaPCf9mUSEMIv6/SaDWFvlXGIBByKIhuyL01LllA/qlSPyj6K0v5XO3dMNitKaUKQEvLFjf40+7Vid48kNfgXKTb6fx6u9Fir9kBjdUDQeAT4bB+F/1Kay+nGZ375eHEyW+Tqc+YUsnuL0hqkR4y7Wu1lm1I185vVpbQ9QWmhiASZYxvoZEEc+4lG2va8pCe1plOo4lPipCLIpTAuSQntwKg+Ry1njfpU8v8LmZm9yO5OePmhEkkDL8HFBpbKgaD+qkufW+XUIDySDlBUJzggPiRGb/ss3XFkYCPd6qzN1LaHYP4aUeeXES/bMdl6C0/XzhvaPIz4/5PkkA0OV1M45U+xKkvuDi1M6Y0= 2021-03-12T13:14:01.074 INFO:teuthology.kill:2021-03-12 13:14:01,069.069 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-12T13:14:01.075 INFO:teuthology.kill:2021-03-12 13:14:01,073.073 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-12T13:14:01.153 INFO:teuthology.kill:2021-03-12 13:14:01,152.152 INFO:teuthology.orchestra.console:Power off smithi092 2021-03-12T13:14:01.178 INFO:teuthology.kill:2021-03-12 13:14:01,178.178 INFO:teuthology.orchestra.console:Power off smithi136 2021-03-12T13:14:09.575 INFO:teuthology.kill:2021-03-12 13:14:09,574.574 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2021-03-12T13:14:13.690 INFO:teuthology.kill:2021-03-12 13:14:13,689.689 INFO:teuthology.orchestra.console:Power off for smithi092 completed 2021-03-12T13:14:13.946 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-03-12T13:14:13.951 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-03-12T13:14:13.951 INFO:teuthology.misc:Compressing logs... 2021-03-12T13:14:13.954 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-12T13:14:48.643 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/teuthology/misc.py", line 1374, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b7dbd64ff8500a97d0659a105ea1a1f415ec323b/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-03-12T13:14:48.711 INFO:teuthology.kill:No teuthology processes running 2021-03-12T13:14:50.718 INFO:teuthology.kill:Nuking machines: ['smithi092', 'smithi136'] 2021-03-12T13:14:51.899 INFO:teuthology.kill:2021-03-12 13:14:51,898.898 INFO:teuthology.nuke:targets: 2021-03-12T13:14:51.900 INFO:teuthology.kill: smithi092.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDyF/RqNcXWcmkiDt+H/IzfM50DNu9ov78V8Z0wuP8Qwny1y1SqpaY+rmZV1cLFz1ItXCs+JYA/7RwKrQiKdpzL4acLY9MyYMagomc6Kv3ishoVh0AwJJBFPvSJ7OlNsHIQF/RV/WpKjiYNOonHhvPbcTMSTfQRCTDDdKd6AuiwdYbzKksmNXdmNYmJ3hqu5Cx1fmUXwU5DQzcJe4mpCigMYTnTy0bqiSzHYCkwyFKWvIiLG0zLdkZF7yaIgz/WSc3PuyXMfZxd9M4OTOSCKD8o2BopUHxnFox4auMZqIihrw1M9tYgrAzmJxI5muaK/XZMDw8OP2R8/tz7D+xUwWuRunQfHNOiBOm/wLwrmaEfiWv6+Uh4dBCNEbvlXft6M2JZUnh1gSRnOToLa9XYbNkGKJ+d4rgH0nYKG/ob9F32q/2GV4Qao8lxccODvc5vOjcraNXSpvoYR9c1t4zcfULLqvn5ggUzfIOycx/WE9lfD/Vag0dQQ8gWlXo4WJ4YD+M= 2021-03-12T13:14:51.900 INFO:teuthology.kill: smithi136.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTOrLsihQJ8eJWuImubvRAl25RXYDPgDSJKzwIGaBPsJjz4p+etAOEF42j6PE8QBLBdCyHFCZ82eW78l5tSBuV8x2kRb9tdrZFwcPF/7K0RK0js3+ci7Ej6svFt2qQb2A9QqctWbdd+C8m0JpqXtGzfUQykSQRaPCf9mUSEMIv6/SaDWFvlXGIBByKIhuyL01LllA/qlSPyj6K0v5XO3dMNitKaUKQEvLFjf40+7Vid48kNfgXKTb6fx6u9Fir9kBjdUDQeAT4bB+F/1Kay+nGZ375eHEyW+Tqc+YUsnuL0hqkR4y7Wu1lm1I185vVpbQ9QWmhiASZYxvoZEEc+4lG2va8pCe1plOo4lPipCLIpTAuSQntwKg+Ry1njfpU8v8LmZm9yO5OePmhEkkDL8HFBpbKgaD+qkufW+XUIDySDlBUJzggPiRGb/ss3XFkYCPd6qzN1LaHYP4aUeeXES/bMdl6C0/XzhvaPIz4/5PkkA0OV1M45U+xKkvuDi1M6Y0= 2021-03-12T13:14:51.921 INFO:teuthology.kill:2021-03-12 13:14:51,921.921 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-12T13:14:51.925 INFO:teuthology.kill:2021-03-12 13:14:51,924.924 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-12T13:14:51.984 INFO:teuthology.kill:2021-03-12 13:14:51,983.983 INFO:teuthology.orchestra.console:Power off smithi136 2021-03-12T13:14:51.997 INFO:teuthology.kill:2021-03-12 13:14:51,997.997 INFO:teuthology.orchestra.console:Power off smithi092 2021-03-12T13:14:52.128 INFO:teuthology.kill:2021-03-12 13:14:52,128.128 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2021-03-12T13:14:52.137 INFO:teuthology.kill:2021-03-12 13:14:52,137.137 INFO:teuthology.orchestra.console:Power off for smithi092 completed 2021-03-12T13:14:52.271 INFO:teuthology.kill:2021-03-12 13:14:52,270.270 INFO:teuthology.lock.ops:unlocked smithi136.front.sepia.ceph.com 2021-03-12T13:14:52.275 INFO:teuthology.kill:2021-03-12 13:14:52,275.275 INFO:teuthology.lock.ops:unlocked smithi092.front.sepia.ceph.com 2021-03-12T13:16:52.577 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-03-12T13:16:52.674 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-03-12T13:16:52.675 INFO:teuthology.nuke:Checking targets against current locks 2021-03-12T13:16:52.763 INFO:teuthology.nuke:Not nuking smithi092.front.sepia.ceph.com because description doesn't match 2021-03-12T13:16:52.763 INFO:teuthology.nuke:Not nuking smithi136.front.sepia.ceph.com because description doesn't match