2021-06-21T14:50:58.720 INFO:root:teuthology version: 1.1.0-f359b10d 2021-06-21T14:50:58.808 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi071.front.sepia.ceph.com' reimaging 2021-06-21T14:50:58.809 INFO:teuthology.lock.ops:Updating [ubuntu@smithi071.front.sepia.ceph.com]: reset os type and version on server 2021-06-21T14:50:58.809 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2021-06-21T14:50:58.822 INFO:teuthology.lock.ops:Node 'ubuntu@smithi071.front.sepia.ceph.com' reimaging is complete 2021-06-21T14:50:58.823 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi109.front.sepia.ceph.com' reimaging 2021-06-21T14:50:58.823 INFO:teuthology.lock.ops:Updating [ubuntu@smithi109.front.sepia.ceph.com]: reset os type and version on server 2021-06-21T14:50:58.823 INFO:teuthology.lock.ops:Updating smithi109.front.sepia.ceph.com on lock server 2021-06-21T14:50:58.838 INFO:teuthology.lock.ops:Node 'ubuntu@smithi109.front.sepia.ceph.com' reimaging is complete 2021-06-21T14:50:59.030 INFO:teuthology.provision.fog.smithi071:Scheduling deploy of centos 8.3 2021-06-21T14:50:59.038 INFO:teuthology.provision.fog.smithi109:Scheduling deploy of centos 8.3 2021-06-21T14:50:59.600 INFO:teuthology.orchestra.console:Power off smithi071 2021-06-21T14:50:59.634 INFO:teuthology.orchestra.console:Power off smithi109 2021-06-21T14:51:12.087 INFO:teuthology.orchestra.console:Power off for smithi071 completed 2021-06-21T14:51:12.104 INFO:teuthology.orchestra.console:Power off for smithi109 completed 2021-06-21T14:51:12.189 INFO:teuthology.orchestra.console:Power on smithi071 2021-06-21T14:51:12.205 INFO:teuthology.orchestra.console:Power on smithi109 2021-06-21T14:51:20.559 INFO:teuthology.orchestra.console:Power on for smithi071 completed 2021-06-21T14:51:20.566 INFO:teuthology.orchestra.console:Power on for smithi109 completed 2021-06-21T14:51:20.661 INFO:teuthology.provision.fog.smithi071:Waiting for deploy to finish 2021-06-21T14:51:20.668 INFO:teuthology.provision.fog.smithi109:Waiting for deploy to finish 2021-06-21T14:55:25.380 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-21T14:55:45.528 INFO:teuthology.provision.fog.smithi071:Node is ready 2021-06-21T14:55:45.551 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071.front.sepia.ceph.com 2021-06-21T14:55:45.607 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71 smithi071.front.sepia.ceph.com smithi071 2021-06-21T14:55:45.956 INFO:teuthology.provision.fog.smithi071:Deploy complete! 2021-06-21T14:55:50.944 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-21T14:55:51.251 INFO:teuthology.provision.fog.smithi109:Node is ready 2021-06-21T14:55:51.265 INFO:teuthology.orchestra.run.smithi109.stdout:smithi109.front.sepia.ceph.com 2021-06-21T14:55:51.319 INFO:teuthology.orchestra.run.smithi109.stdout:172.21.15.109 smithi109.front.sepia.ceph.com smithi109 2021-06-21T14:55:51.640 INFO:teuthology.provision.fog.smithi109:Deploy complete! 2021-06-21T14:55:51.753 INFO:teuthology.lock.ops:Checking smithi071.front.sepia.ceph.com 2021-06-21T14:55:51.754 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-21T14:55:51.768 INFO:teuthology.lock.ops:Checking smithi109.front.sepia.ceph.com 2021-06-21T14:55:51.769 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-21T14:55:51.783 INFO:teuthology.lock.ops:Updating [smithi071.front.sepia.ceph.com]: set os type and version on server 2021-06-21T14:55:51.783 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-21T14:55:52.070 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2021-06-21T14:55:52.125 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Linux" 2021-06-21T14:55:52.125 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="8" 2021-06-21T14:55:52.125 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2021-06-21T14:55:52.125 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2021-06-21T14:55:52.126 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="8" 2021-06-21T14:55:52.126 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el8" 2021-06-21T14:55:52.126 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Linux 8" 2021-06-21T14:55:52.126 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2021-06-21T14:55:52.126 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-06-21T14:55:52.127 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2021-06-21T14:55:52.127 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-06-21T14:55:52.127 INFO:teuthology.orchestra.run.smithi071.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-06-21T14:55:52.127 INFO:teuthology.orchestra.run.smithi071.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-06-21T14:55:52.128 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2021-06-21T14:55:52.146 INFO:teuthology.lock.ops:Updating [smithi109.front.sepia.ceph.com]: set os type and version on server 2021-06-21T14:55:52.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-21T14:55:52.422 INFO:teuthology.orchestra.run.smithi109.stdout:x86_64 2021-06-21T14:55:52.476 INFO:teuthology.orchestra.run.smithi109.stdout:NAME="CentOS Linux" 2021-06-21T14:55:52.477 INFO:teuthology.orchestra.run.smithi109.stdout:VERSION="8" 2021-06-21T14:55:52.477 INFO:teuthology.orchestra.run.smithi109.stdout:ID="centos" 2021-06-21T14:55:52.477 INFO:teuthology.orchestra.run.smithi109.stdout:ID_LIKE="rhel fedora" 2021-06-21T14:55:52.477 INFO:teuthology.orchestra.run.smithi109.stdout:VERSION_ID="8" 2021-06-21T14:55:52.478 INFO:teuthology.orchestra.run.smithi109.stdout:PLATFORM_ID="platform:el8" 2021-06-21T14:55:52.478 INFO:teuthology.orchestra.run.smithi109.stdout:PRETTY_NAME="CentOS Linux 8" 2021-06-21T14:55:52.478 INFO:teuthology.orchestra.run.smithi109.stdout:ANSI_COLOR="0;31" 2021-06-21T14:55:52.478 INFO:teuthology.orchestra.run.smithi109.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-06-21T14:55:52.479 INFO:teuthology.orchestra.run.smithi109.stdout:HOME_URL="https://centos.org/" 2021-06-21T14:55:52.479 INFO:teuthology.orchestra.run.smithi109.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-06-21T14:55:52.479 INFO:teuthology.orchestra.run.smithi109.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-06-21T14:55:52.479 INFO:teuthology.orchestra.run.smithi109.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-06-21T14:55:52.480 INFO:teuthology.lock.ops:Updating smithi109.front.sepia.ceph.com on lock server 2021-06-21T14:55:52.549 INFO:teuthology.dispatcher.supervisor:Running job 6182678 2021-06-21T14:55:52.549 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology -v --owner scheduled_rzarzynski@teuthology --archive /home/teuthworker/archive/rzarzynski-2021-06-21_14:49:36-rados-master-distro-basic-smithi/6182678 --name rzarzynski-2021-06-21_14:49:36-rados-master-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/small-objects-balanced} -- /home/teuthworker/archive/rzarzynski-2021-06-21_14:49:36-rados-master-distro-basic-smithi/6182678/orig.config.yaml 2021-06-21T14:55:52.552 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rzarzynski-2021-06-21_14:49:36-rados-master-distro-basic-smithi/6182678 2021-06-21T14:55:52.553 INFO:teuthology.dispatcher.supervisor:Job PID: 28586 2021-06-21T14:55:52.553 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-06-22T02:56:45.859 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-06-22T02:56:47.317 INFO:teuthology.kill:Killing Pids: {28586} 2021-06-22T02:56:48.658 INFO:teuthology.kill:Nuking machines: ['smithi071', 'smithi109'] 2021-06-22T02:56:49.609 INFO:teuthology.kill:2021-06-22 02:56:49,609.609 INFO:teuthology.nuke:targets: 2021-06-22T02:56:49.611 INFO:teuthology.kill: smithi071.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhgmrSUH9JQU4On91JwHSuTfFMycQ3ENQxmIQo4l3DQaj6jzxM6dnq42Cty74e31rQeZAa0TF098ZM9lx2HULTonfdUeyezun6QT02YWy2pKocY6SutsLQmXi2avZIHTJN1yFcdhbjdf6/0HAoZYVQ1QXfbce51lEw15p5PEUf4s6UDjWfVK4nP0ZEnGXjyYHSjbTQ32ZHbvyqEoeQGS3xq+afVo+ZVYkQpUf6lbVoaClAWt4pgcGEVynLgserVVL/BrdPb6ZraSdzgMkOSE4LSvll6iYV7lih7pW+lxOWPiWCiY7Q4smjsah8wZ6hCYHUQKpsyept3dWPCIPnMP0Ky7qK505eJPSJFCb5AAJVDpqnM4vhFZHdDt2+20CKqrz2CsUNEB4hM7NsH5uvIFIQFyWmzkLk8njNvA6RR6+5mPLNr6/TbWWdiYHYRbntaMMVfyju0O+pcnNv9mZJJeBec2xuZZaEEZ6tyzP/d7Ql3bGF2a077MmFWIUJfiVPUm8= 2021-06-22T02:56:49.612 INFO:teuthology.kill: smithi109.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZLC5WtpXIYIFvCYkhMmSWroP+KyLZxAnDWLVCATyL1DoV1BWOn7+fTOcwLkkIozbP5zaeJ21J4FIRelYOMUzxpvq9Q7mkHxsGb+pw+F8hby/1TVrHPZCKmni6DvNIh8zeS3VShxkuDdX49MiGjtpZdra0dLyrMsK5ifv2SW+ZXo8olQVq0cB8mXKhUr/NBkQAu+WbK7u8pDdKlQh/ldNXDuQ13oBDOiynuZfOoB75We2+auHLPvOSod35PFM0E3DLLWrdMamiIF7fy8Hny+pLAB6nZYj8gLiuf503IqIcKIRXBZu83xboO3I0Gm/Zu+vWrc46YyQ5HOEFK3Y9ZG7Ytuu+mjz/JRBlKkrvjvfWcBHmh1yrwNpoPLSIfI/nAAovNywmlMggZ8GrW/63zwH5eWHqTlVudYuX6f/68VAzKzWOHfZnlWbkxK0TS6+wqklWItGfIRa5og9v9R632fU1TcfgbNgzE37sJCp1uZxiFd8+rDFPzK6z63ry/21KbHU= 2021-06-22T02:56:49.613 INFO:teuthology.kill:2021-06-22 02:56:49,609.609 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-22T02:56:49.614 INFO:teuthology.kill:2021-06-22 02:56:49,612.612 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-22T02:56:49.663 INFO:teuthology.kill:2021-06-22 02:56:49,663.663 INFO:teuthology.orchestra.console:Power off smithi071 2021-06-22T02:56:49.678 INFO:teuthology.kill:2021-06-22 02:56:49,678.678 INFO:teuthology.orchestra.console:Power off smithi109 2021-06-22T02:57:02.144 INFO:teuthology.kill:2021-06-22 02:57:02,143.143 INFO:teuthology.orchestra.console:Power off for smithi071 completed 2021-06-22T02:57:02.150 INFO:teuthology.kill:2021-06-22 02:57:02,150.150 INFO:teuthology.orchestra.console:Power off for smithi109 completed 2021-06-22T02:57:02.611 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-06-22T02:57:02.614 INFO:teuthology.task.internal:roles: ubuntu@smithi109.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-06-22T02:57:02.618 INFO:teuthology.misc:Compressing logs... 2021-06-22T02:57:02.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-22T02:57:37.244 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-06-22T02:57:37.336 INFO:teuthology.kill:No teuthology processes running 2021-06-22T02:57:38.442 INFO:teuthology.kill:Nuking machines: ['smithi071', 'smithi109'] 2021-06-22T02:57:39.375 INFO:teuthology.kill:2021-06-22 02:57:39,374.374 INFO:teuthology.nuke:targets: 2021-06-22T02:57:39.376 INFO:teuthology.kill: smithi071.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhgmrSUH9JQU4On91JwHSuTfFMycQ3ENQxmIQo4l3DQaj6jzxM6dnq42Cty74e31rQeZAa0TF098ZM9lx2HULTonfdUeyezun6QT02YWy2pKocY6SutsLQmXi2avZIHTJN1yFcdhbjdf6/0HAoZYVQ1QXfbce51lEw15p5PEUf4s6UDjWfVK4nP0ZEnGXjyYHSjbTQ32ZHbvyqEoeQGS3xq+afVo+ZVYkQpUf6lbVoaClAWt4pgcGEVynLgserVVL/BrdPb6ZraSdzgMkOSE4LSvll6iYV7lih7pW+lxOWPiWCiY7Q4smjsah8wZ6hCYHUQKpsyept3dWPCIPnMP0Ky7qK505eJPSJFCb5AAJVDpqnM4vhFZHdDt2+20CKqrz2CsUNEB4hM7NsH5uvIFIQFyWmzkLk8njNvA6RR6+5mPLNr6/TbWWdiYHYRbntaMMVfyju0O+pcnNv9mZJJeBec2xuZZaEEZ6tyzP/d7Ql3bGF2a077MmFWIUJfiVPUm8= 2021-06-22T02:57:39.376 INFO:teuthology.kill: smithi109.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZLC5WtpXIYIFvCYkhMmSWroP+KyLZxAnDWLVCATyL1DoV1BWOn7+fTOcwLkkIozbP5zaeJ21J4FIRelYOMUzxpvq9Q7mkHxsGb+pw+F8hby/1TVrHPZCKmni6DvNIh8zeS3VShxkuDdX49MiGjtpZdra0dLyrMsK5ifv2SW+ZXo8olQVq0cB8mXKhUr/NBkQAu+WbK7u8pDdKlQh/ldNXDuQ13oBDOiynuZfOoB75We2+auHLPvOSod35PFM0E3DLLWrdMamiIF7fy8Hny+pLAB6nZYj8gLiuf503IqIcKIRXBZu83xboO3I0Gm/Zu+vWrc46YyQ5HOEFK3Y9ZG7Ytuu+mjz/JRBlKkrvjvfWcBHmh1yrwNpoPLSIfI/nAAovNywmlMggZ8GrW/63zwH5eWHqTlVudYuX6f/68VAzKzWOHfZnlWbkxK0TS6+wqklWItGfIRa5og9v9R632fU1TcfgbNgzE37sJCp1uZxiFd8+rDFPzK6z63ry/21KbHU= 2021-06-22T02:57:39.396 INFO:teuthology.kill:2021-06-22 02:57:39,396.396 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-22T02:57:39.400 INFO:teuthology.kill:2021-06-22 02:57:39,400.400 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-22T02:57:39.464 INFO:teuthology.kill:2021-06-22 02:57:39,464.464 INFO:teuthology.orchestra.console:Power off smithi071 2021-06-22T02:57:39.477 INFO:teuthology.kill:2021-06-22 02:57:39,477.477 INFO:teuthology.orchestra.console:Power off smithi109 2021-06-22T02:57:39.611 INFO:teuthology.kill:2021-06-22 02:57:39,610.610 INFO:teuthology.orchestra.console:Power off for smithi071 completed 2021-06-22T02:57:39.615 INFO:teuthology.kill:2021-06-22 02:57:39,615.615 INFO:teuthology.orchestra.console:Power off for smithi109 completed 2021-06-22T02:57:39.761 INFO:teuthology.kill:2021-06-22 02:57:39,760.760 INFO:teuthology.lock.ops:unlocked smithi071.front.sepia.ceph.com 2021-06-22T02:57:39.763 INFO:teuthology.kill:2021-06-22 02:57:39,763.763 INFO:teuthology.lock.ops:unlocked smithi109.front.sepia.ceph.com 2021-06-22T02:59:40.062 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-06-22T02:59:40.240 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-06-22T02:59:40.243 INFO:teuthology.nuke:Checking targets against current locks 2021-06-22T02:59:40.466 INFO:teuthology.nuke:Not nuking smithi071.front.sepia.ceph.com because description doesn't match 2021-06-22T02:59:40.467 INFO:teuthology.nuke:Not nuking smithi109.front.sepia.ceph.com because description doesn't match