2021-09-02T11:47:30.873 INFO:root:teuthology version: 1.1.0-df69d6db 2021-09-02T11:47:32.027 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi172.front.sepia.ceph.com' reimaging 2021-09-02T11:47:32.028 INFO:teuthology.lock.ops:Updating [ubuntu@smithi172.front.sepia.ceph.com]: reset os type and version on server 2021-09-02T11:47:32.029 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2021-09-02T11:47:32.046 INFO:teuthology.lock.ops:Node 'ubuntu@smithi172.front.sepia.ceph.com' reimaging is complete 2021-09-02T11:47:32.047 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi184.front.sepia.ceph.com' reimaging 2021-09-02T11:47:32.047 INFO:teuthology.lock.ops:Updating [ubuntu@smithi184.front.sepia.ceph.com]: reset os type and version on server 2021-09-02T11:47:32.047 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2021-09-02T11:47:32.068 INFO:teuthology.lock.ops:Node 'ubuntu@smithi184.front.sepia.ceph.com' reimaging is complete 2021-09-02T11:47:32.257 INFO:teuthology.provision.fog.smithi172:Scheduling deploy of centos 8.3 2021-09-02T11:47:32.279 INFO:teuthology.provision.fog.smithi184:Scheduling deploy of centos 8.3 2021-09-02T11:47:32.661 INFO:teuthology.orchestra.console:Power off smithi172 2021-09-02T11:47:32.737 INFO:teuthology.orchestra.console:Power off smithi184 2021-09-02T11:47:41.033 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2021-09-02T11:47:41.105 INFO:teuthology.orchestra.console:Power off for smithi184 completed 2021-09-02T11:47:41.135 INFO:teuthology.orchestra.console:Power on smithi172 2021-09-02T11:47:41.207 INFO:teuthology.orchestra.console:Power on smithi184 2021-09-02T11:47:49.504 INFO:teuthology.orchestra.console:Power on for smithi172 completed 2021-09-02T11:47:49.569 INFO:teuthology.orchestra.console:Power on for smithi184 completed 2021-09-02T11:47:49.607 INFO:teuthology.provision.fog.smithi172:Waiting for deploy to finish 2021-09-02T11:47:49.670 INFO:teuthology.provision.fog.smithi184:Waiting for deploy to finish 2021-09-02T11:52:21.845 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-09-02T11:52:22.140 INFO:teuthology.provision.fog.smithi184:Node is ready 2021-09-02T11:52:22.155 INFO:teuthology.orchestra.run.smithi184.stdout:smithi184.front.sepia.ceph.com 2021-09-02T11:52:22.210 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.15.184 smithi184.front.sepia.ceph.com smithi184 2021-09-02T11:52:22.532 INFO:teuthology.provision.fog.smithi184:Deploy complete! 2021-09-02T11:52:46.193 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-09-02T11:52:46.491 INFO:teuthology.provision.fog.smithi172:Node is ready 2021-09-02T11:52:46.507 INFO:teuthology.orchestra.run.smithi172.stdout:smithi172.front.sepia.ceph.com 2021-09-02T11:52:46.563 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172 smithi172.front.sepia.ceph.com smithi172 2021-09-02T11:52:46.897 INFO:teuthology.provision.fog.smithi172:Deploy complete! 2021-09-02T11:52:47.042 INFO:teuthology.lock.ops:Checking smithi172.front.sepia.ceph.com 2021-09-02T11:52:47.043 INFO:teuthology.lock.ops:New key found. Updating... 2021-09-02T11:52:47.060 INFO:teuthology.lock.ops:Checking smithi184.front.sepia.ceph.com 2021-09-02T11:52:47.060 INFO:teuthology.lock.ops:New key found. Updating... 2021-09-02T11:52:47.077 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: set os type and version on server 2021-09-02T11:52:47.078 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-09-02T11:52:47.330 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2021-09-02T11:52:47.384 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Linux" 2021-09-02T11:52:47.385 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="8" 2021-09-02T11:52:47.385 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2021-09-02T11:52:47.385 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2021-09-02T11:52:47.386 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="8" 2021-09-02T11:52:47.386 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el8" 2021-09-02T11:52:47.387 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Linux 8" 2021-09-02T11:52:47.388 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2021-09-02T11:52:47.388 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-09-02T11:52:47.388 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2021-09-02T11:52:47.388 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-09-02T11:52:47.389 INFO:teuthology.orchestra.run.smithi172.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-09-02T11:52:47.389 INFO:teuthology.orchestra.run.smithi172.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-09-02T11:52:47.390 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2021-09-02T11:52:47.405 INFO:teuthology.lock.ops:Updating [smithi184.front.sepia.ceph.com]: set os type and version on server 2021-09-02T11:52:47.406 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-09-02T11:52:47.668 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2021-09-02T11:52:47.723 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Linux" 2021-09-02T11:52:47.723 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="8" 2021-09-02T11:52:47.723 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2021-09-02T11:52:47.724 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2021-09-02T11:52:47.724 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="8" 2021-09-02T11:52:47.724 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el8" 2021-09-02T11:52:47.724 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Linux 8" 2021-09-02T11:52:47.725 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2021-09-02T11:52:47.725 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-09-02T11:52:47.726 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2021-09-02T11:52:47.726 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-09-02T11:52:47.726 INFO:teuthology.orchestra.run.smithi184.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-09-02T11:52:47.726 INFO:teuthology.orchestra.run.smithi184.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-09-02T11:52:47.727 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2021-09-02T11:52:47.808 INFO:teuthology.dispatcher.supervisor:Running job 6372429 2021-09-02T11:52:47.808 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/virtualenv/bin/teuthology -v --owner scheduled_rzarzynski@teuthology --archive /home/teuthworker/archive/rzarzynski-2021-09-02_11:40:36-rados-master-distro-basic-smithi/6372429 --name rzarzynski-2021-09-02_11:40:36-rados-master-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/redirect_promote_tests} -- /home/teuthworker/archive/rzarzynski-2021-09-02_11:40:36-rados-master-distro-basic-smithi/6372429/orig.config.yaml 2021-09-02T11:52:47.812 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rzarzynski-2021-09-02_11:40:36-rados-master-distro-basic-smithi/6372429 2021-09-02T11:52:47.813 INFO:teuthology.dispatcher.supervisor:Job PID: 25745 2021-09-02T11:52:47.814 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-09-02T23:53:34.793 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-09-02T23:53:34.973 INFO:teuthology.kill:Killing Pids: {25745} 2021-09-02T23:53:36.089 INFO:teuthology.kill:Nuking machines: ['smithi172', 'smithi184'] 2021-09-02T23:53:37.044 INFO:teuthology.kill:2021-09-02 23:53:37,043.043 INFO:teuthology.nuke:targets: 2021-09-02T23:53:37.045 INFO:teuthology.kill: smithi172.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCnEpdwTbhwDGlC0DmNKxWrx/hAoFfy6J8FiigwbGj84tch7Rw6hXFfX9hkfZS45+OX6PM6EQ1dx3HmPQkUe8b7vlouWMkUctJAqKlfB0RWQvnvv+YkShxu6K1FIHELLVxLt4wC9nJHK3YqmQ5xQx4vsF4RJfsHW0NpLHQ0dhAY0RHlzSf7SaT5+0le2CFebQsJoAmkbsqywN/BK5GAcn8WeXwc9kIAsc+b8b1BhkSimRKDaMO0sYCWWJN0qibo0r1a7u/1ovDxnZrzcaZmtRroSLzjdb1E+qZoT4PgciXHmy9eXP8a/pyv23RpzQWtfFUDOqG9kaXCcBrNuazNm0gH1BiExA4Pz/sNFNx0s435WH7QWsaMjVFD/gWpB7S4vBAaWzohONja1SmJjnKzsRRDYsRffYkqxSTWGsBShHxc/Ks+opHQPIt5skzMVOxOIXSlnG1uK5sLZnjKuM6Jmb43/8yZ4a0seSlJot1NGckEbq6EC9fblbJzBrpwnJjFKVU= 2021-09-02T23:53:37.045 INFO:teuthology.kill: smithi184.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMAkuNB4soJe9MyGtcJAuh1w+QVWkY/xopATt6VLUWCsji+yEGm7AuTCx1EI0/74X6otYoPPuXq8oDP9EH3vke1D4xOWVFdsZnZNSVcPfDDtxnDYmXePXbbbLYZjiPaSv7gXF8rrRpRLkiy7Nw6UoHfKQqNG1v00nN3nJa4FHEeq2M0OEOd/5m29CGC8mdGpsiGRCrVga+FwV2iiB+r1zGIxUHzJdEwZ7eC04Apd7hDNrDVLTfCWLBacLrDuOOc8bJ7X/up9uNvpCuzb5xouIP/XhIPgKikWPrkLCADMA34igSlZSwDJJXYQ6Cnt8SSHdv6A5XB7sFmbFn0FRDZOEdWHF2Vs731c36w5HgFWndVRgQZXm+bI2birmpKR3EsrIKVPpuoHUliUmXAV+h553+P+HyXR7877bEFnR0/7fvx61eFRnkh8+FnRkTPKdS9+ycH2LJlVBuCR469g0zk8flB76LosIIO3IPmN2nead20IbVKg8e+zuipAcEapO4w6c= 2021-09-02T23:53:37.045 INFO:teuthology.kill:2021-09-02 23:53:37,044.044 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-02T23:53:37.047 INFO:teuthology.kill:2021-09-02 23:53:37,046.046 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-02T23:53:37.110 INFO:teuthology.kill:2021-09-02 23:53:37,110.110 INFO:teuthology.orchestra.console:Power off smithi172 2021-09-02T23:53:37.127 INFO:teuthology.kill:2021-09-02 23:53:37,127.127 INFO:teuthology.orchestra.console:Power off smithi184 2021-09-02T23:53:49.599 INFO:teuthology.kill:2021-09-02 23:53:49,599.599 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2021-09-02T23:53:49.604 INFO:teuthology.kill:2021-09-02 23:53:49,603.603 INFO:teuthology.orchestra.console:Power off for smithi184 completed 2021-09-02T23:53:49.828 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-09-02T23:53:49.829 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-09-02T23:53:49.829 INFO:teuthology.misc:Compressing logs... 2021-09-02T23:53:49.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-09-02T23:54:49.878 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_df69d6db9d68f5492f47c1dee57bca3b0a385796/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-09-02T23:54:49.932 INFO:teuthology.kill:No teuthology processes running 2021-09-02T23:54:50.964 INFO:teuthology.kill:Nuking machines: ['smithi172', 'smithi184'] 2021-09-02T23:54:51.839 INFO:teuthology.kill:2021-09-02 23:54:51,839.839 INFO:teuthology.nuke:targets: 2021-09-02T23:54:51.840 INFO:teuthology.kill: smithi172.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCnEpdwTbhwDGlC0DmNKxWrx/hAoFfy6J8FiigwbGj84tch7Rw6hXFfX9hkfZS45+OX6PM6EQ1dx3HmPQkUe8b7vlouWMkUctJAqKlfB0RWQvnvv+YkShxu6K1FIHELLVxLt4wC9nJHK3YqmQ5xQx4vsF4RJfsHW0NpLHQ0dhAY0RHlzSf7SaT5+0le2CFebQsJoAmkbsqywN/BK5GAcn8WeXwc9kIAsc+b8b1BhkSimRKDaMO0sYCWWJN0qibo0r1a7u/1ovDxnZrzcaZmtRroSLzjdb1E+qZoT4PgciXHmy9eXP8a/pyv23RpzQWtfFUDOqG9kaXCcBrNuazNm0gH1BiExA4Pz/sNFNx0s435WH7QWsaMjVFD/gWpB7S4vBAaWzohONja1SmJjnKzsRRDYsRffYkqxSTWGsBShHxc/Ks+opHQPIt5skzMVOxOIXSlnG1uK5sLZnjKuM6Jmb43/8yZ4a0seSlJot1NGckEbq6EC9fblbJzBrpwnJjFKVU= 2021-09-02T23:54:51.840 INFO:teuthology.kill: smithi184.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMAkuNB4soJe9MyGtcJAuh1w+QVWkY/xopATt6VLUWCsji+yEGm7AuTCx1EI0/74X6otYoPPuXq8oDP9EH3vke1D4xOWVFdsZnZNSVcPfDDtxnDYmXePXbbbLYZjiPaSv7gXF8rrRpRLkiy7Nw6UoHfKQqNG1v00nN3nJa4FHEeq2M0OEOd/5m29CGC8mdGpsiGRCrVga+FwV2iiB+r1zGIxUHzJdEwZ7eC04Apd7hDNrDVLTfCWLBacLrDuOOc8bJ7X/up9uNvpCuzb5xouIP/XhIPgKikWPrkLCADMA34igSlZSwDJJXYQ6Cnt8SSHdv6A5XB7sFmbFn0FRDZOEdWHF2Vs731c36w5HgFWndVRgQZXm+bI2birmpKR3EsrIKVPpuoHUliUmXAV+h553+P+HyXR7877bEFnR0/7fvx61eFRnkh8+FnRkTPKdS9+ycH2LJlVBuCR469g0zk8flB76LosIIO3IPmN2nead20IbVKg8e+zuipAcEapO4w6c= 2021-09-02T23:54:51.861 INFO:teuthology.kill:2021-09-02 23:54:51,861.861 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-02T23:54:51.864 INFO:teuthology.kill:2021-09-02 23:54:51,864.864 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-09-02T23:54:51.919 INFO:teuthology.kill:2021-09-02 23:54:51,919.919 INFO:teuthology.orchestra.console:Power off smithi184 2021-09-02T23:54:51.932 INFO:teuthology.kill:2021-09-02 23:54:51,931.931 INFO:teuthology.orchestra.console:Power off smithi172 2021-09-02T23:54:52.059 INFO:teuthology.kill:2021-09-02 23:54:52,059.059 INFO:teuthology.orchestra.console:Power off for smithi184 completed 2021-09-02T23:54:52.069 INFO:teuthology.kill:2021-09-02 23:54:52,069.069 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2021-09-02T23:54:52.207 INFO:teuthology.kill:2021-09-02 23:54:52,206.206 INFO:teuthology.lock.ops:unlocked smithi184.front.sepia.ceph.com 2021-09-02T23:54:52.210 INFO:teuthology.kill:2021-09-02 23:54:52,210.210 INFO:teuthology.lock.ops:unlocked smithi172.front.sepia.ceph.com 2021-09-02T23:56:52.491 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-09-02T23:56:52.597 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-09-02T23:56:52.598 INFO:teuthology.nuke:Checking targets against current locks 2021-09-02T23:56:52.723 INFO:teuthology.nuke:Not nuking smithi184.front.sepia.ceph.com because description doesn't match 2021-09-02T23:56:52.724 INFO:teuthology.nuke:Not nuking smithi172.front.sepia.ceph.com because description doesn't match