2021-04-23T00:19:02.425 INFO:root:teuthology version: 1.1.0-2713a3cd 2021-04-23T00:19:02.533 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi140.front.sepia.ceph.com' reimaging 2021-04-23T00:19:02.533 INFO:teuthology.lock.ops:Updating [ubuntu@smithi140.front.sepia.ceph.com]: reset os type and version on server 2021-04-23T00:19:02.534 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2021-04-23T00:19:02.552 INFO:teuthology.lock.ops:Node 'ubuntu@smithi140.front.sepia.ceph.com' reimaging is complete 2021-04-23T00:19:02.552 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi196.front.sepia.ceph.com' reimaging 2021-04-23T00:19:02.553 INFO:teuthology.lock.ops:Updating [ubuntu@smithi196.front.sepia.ceph.com]: reset os type and version on server 2021-04-23T00:19:02.553 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2021-04-23T00:19:02.568 INFO:teuthology.lock.ops:Node 'ubuntu@smithi196.front.sepia.ceph.com' reimaging is complete 2021-04-23T00:19:02.795 INFO:teuthology.provision.fog.smithi140:Scheduling deploy of centos 8.2 2021-04-23T00:19:02.826 INFO:teuthology.provision.fog.smithi196:Scheduling deploy of centos 8.2 2021-04-23T00:19:03.282 INFO:teuthology.orchestra.console:Power off smithi140 2021-04-23T00:19:03.332 INFO:teuthology.orchestra.console:Power off smithi196 2021-04-23T00:19:11.695 INFO:teuthology.orchestra.console:Power off for smithi140 completed 2021-04-23T00:19:11.796 INFO:teuthology.orchestra.console:Power on smithi140 2021-04-23T00:19:15.835 INFO:teuthology.orchestra.console:Power off for smithi196 completed 2021-04-23T00:19:15.937 INFO:teuthology.orchestra.console:Power on smithi196 2021-04-23T00:19:20.160 INFO:teuthology.orchestra.console:Power on for smithi140 completed 2021-04-23T00:19:20.190 INFO:teuthology.orchestra.console:Power on for smithi196 completed 2021-04-23T00:19:20.261 INFO:teuthology.provision.fog.smithi140:Waiting for deploy to finish 2021-04-23T00:19:20.291 INFO:teuthology.provision.fog.smithi196:Waiting for deploy to finish 2021-04-23T00:23:52.173 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-04-23T00:23:52.491 INFO:teuthology.provision.fog.smithi196:Node is ready 2021-04-23T00:23:52.505 INFO:teuthology.orchestra.run.smithi196.stdout:smithi196.front.sepia.ceph.com 2021-04-23T00:23:52.559 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.196 smithi196.front.sepia.ceph.com smithi196 2021-04-23T00:23:52.898 INFO:teuthology.provision.fog.smithi196:Deploy complete! 2021-04-23T00:24:16.300 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-04-23T00:24:16.594 INFO:teuthology.provision.fog.smithi140:Node is ready 2021-04-23T00:24:16.608 INFO:teuthology.orchestra.run.smithi140.stdout:smithi140.front.sepia.ceph.com 2021-04-23T00:24:16.662 INFO:teuthology.orchestra.run.smithi140.stdout:172.21.15.140 smithi140.front.sepia.ceph.com smithi140 2021-04-23T00:24:16.987 INFO:teuthology.provision.fog.smithi140:Deploy complete! 2021-04-23T00:24:17.108 INFO:teuthology.lock.ops:Checking smithi140.front.sepia.ceph.com 2021-04-23T00:24:17.109 INFO:teuthology.lock.ops:New key found. Updating... 2021-04-23T00:24:17.125 INFO:teuthology.lock.ops:Checking smithi196.front.sepia.ceph.com 2021-04-23T00:24:17.125 INFO:teuthology.lock.ops:New key found. Updating... 2021-04-23T00:24:17.139 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: set os type and version on server 2021-04-23T00:24:17.140 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-23T00:24:17.402 INFO:teuthology.orchestra.run.smithi140.stdout:x86_64 2021-04-23T00:24:17.623 INFO:teuthology.orchestra.run.smithi140.stdout:NAME="CentOS Linux" 2021-04-23T00:24:17.623 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION="8 (Core)" 2021-04-23T00:24:17.623 INFO:teuthology.orchestra.run.smithi140.stdout:ID="centos" 2021-04-23T00:24:17.623 INFO:teuthology.orchestra.run.smithi140.stdout:ID_LIKE="rhel fedora" 2021-04-23T00:24:17.624 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION_ID="8" 2021-04-23T00:24:17.624 INFO:teuthology.orchestra.run.smithi140.stdout:PLATFORM_ID="platform:el8" 2021-04-23T00:24:17.624 INFO:teuthology.orchestra.run.smithi140.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-04-23T00:24:17.624 INFO:teuthology.orchestra.run.smithi140.stdout:ANSI_COLOR="0;31" 2021-04-23T00:24:17.624 INFO:teuthology.orchestra.run.smithi140.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-04-23T00:24:17.624 INFO:teuthology.orchestra.run.smithi140.stdout:HOME_URL="https://www.centos.org/" 2021-04-23T00:24:17.624 INFO:teuthology.orchestra.run.smithi140.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-04-23T00:24:17.624 INFO:teuthology.orchestra.run.smithi140.stdout: 2021-04-23T00:24:17.624 INFO:teuthology.orchestra.run.smithi140.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-04-23T00:24:17.624 INFO:teuthology.orchestra.run.smithi140.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-04-23T00:24:17.624 INFO:teuthology.orchestra.run.smithi140.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-04-23T00:24:17.624 INFO:teuthology.orchestra.run.smithi140.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-04-23T00:24:17.624 INFO:teuthology.orchestra.run.smithi140.stdout: 2021-04-23T00:24:17.626 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2021-04-23T00:24:17.642 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: set os type and version on server 2021-04-23T00:24:17.642 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-23T00:24:17.912 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2021-04-23T00:24:17.966 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Linux" 2021-04-23T00:24:17.966 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="8 (Core)" 2021-04-23T00:24:17.966 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2021-04-23T00:24:17.966 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2021-04-23T00:24:17.966 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="8" 2021-04-23T00:24:17.966 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el8" 2021-04-23T00:24:17.966 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-04-23T00:24:17.966 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2021-04-23T00:24:17.966 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-04-23T00:24:17.966 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://www.centos.org/" 2021-04-23T00:24:17.967 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-04-23T00:24:17.967 INFO:teuthology.orchestra.run.smithi196.stdout: 2021-04-23T00:24:17.967 INFO:teuthology.orchestra.run.smithi196.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-04-23T00:24:17.967 INFO:teuthology.orchestra.run.smithi196.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-04-23T00:24:17.967 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-04-23T00:24:17.967 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-04-23T00:24:17.967 INFO:teuthology.orchestra.run.smithi196.stdout: 2021-04-23T00:24:17.968 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2021-04-23T00:24:18.050 INFO:teuthology.dispatcher.supervisor:Running job 6065751 2021-04-23T00:24:18.050 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-04-22_19:53:40-rados-wip-yuri4-testing-2021-04-22-1018-pacific-distro-basic-smithi/6065751 --name yuriw-2021-04-22_19:53:40-rados-wip-yuri4-testing-2021-04-22-1018-pacific-distro-basic-smithi --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} -- /home/teuthworker/archive/yuriw-2021-04-22_19:53:40-rados-wip-yuri4-testing-2021-04-22-1018-pacific-distro-basic-smithi/6065751/orig.config.yaml 2021-04-23T00:24:18.055 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-04-22_19:53:40-rados-wip-yuri4-testing-2021-04-22-1018-pacific-distro-basic-smithi/6065751 2021-04-23T00:24:18.055 INFO:teuthology.dispatcher.supervisor:Job PID: 12628 2021-04-23T00:24:18.055 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-04-23T12:25:11.768 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-04-23T12:25:11.982 INFO:teuthology.kill:Killing Pids: {12628} 2021-04-23T12:25:13.145 INFO:teuthology.kill:Nuking machines: ['smithi140', 'smithi196'] 2021-04-23T12:25:14.066 INFO:teuthology.kill:2021-04-23 12:25:14,065.065 INFO:teuthology.nuke:targets: 2021-04-23T12:25:14.066 INFO:teuthology.kill: smithi140.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJksBhRn5cLwRXEDUwgFWwVoQMukJbOcF2rJNRKszbOlT7DO/r/YwJ8ADFk4rDdnuOsAU5Fou8EY9jkvBZPN9nYZM++dFZy4muky19iNjoVWHJDF/ECssel5KGE1CAutBMIOp5aqZ5SqvM70Qazm4TEZfoFaTq26L7lOqjQ+NywkZ6AUK2TZsy9nm+JubSO7sApmu6IuphMzPjU3AMfNzfOwCzgmhMuG1Ubq8cZqHD8Z4TnXxc3wTtguSvkhkzknQuzxQOhRW/Vk3q5gE6AEyL4gCv0EJwWfCBHDFvj304IsywjWdAmB4kHcJ3bKSIBaXBBvUfL7XpMk0JOQgjLySBglTeDKcHXTHjWwuilD8iWJ+y+pJNdaMlecqQlJ2IbiY8+y5EBPh7dvymjDda2pG0MZsJV2utOQnT4nrL8NQG2Oq/sjhS2rodm/Sscqn6Cw0RL3emjUIK22H6pTAdJUTRccIddbA8M6Yh6WBaoRcQ2aOf2YKJqk+sbJ/LdS5mSS0= 2021-04-23T12:25:14.066 INFO:teuthology.kill: smithi196.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0Xh14rznrNp3onjl7P/NnFSzFW/OdIBw9ufFHtCcqVnCyVtfrMn4oB6m0SAebL4AIePrzSUCzNRvzGBSwjAybKVBuYhO13IUCDlJViYQKxgW77j+BN5ajaQwvkQG9nEDWmOM+UsRiqLTmBZ/Z6zrbP0Qd8MGWe2ePvfnlgMJU+dil27LEz4VwZ2v4T2DtY4nqlf3WbYEBRyKKVW9S1hF4cvkmOJ6NDZauem5pj69xezbSMWMIQLhO5MirqWBZlt8llpntXzzg3KWyMK3RGmzzSRZHXvLIOH8A7wx6a+Z25P4ZKytPxthKjvMr4sEaGgCtnYIz5OWtZIu9lnLvkef7wmSeWQjlRrIQJRR2VaNZ5X+DEWNosnkXbF2Tgq7b97QhC0pmWCVOfI0XYNqjAsqO/GcgoHVzKfVg+v7w323WQFgHViHyhCUPvliLm8emN41VeGfYE49ec64fmOkPXxKTN1IOIOT0YFaHi7alCl4qCx/1Y9QhJRSO7YySXMVF+gM= 2021-04-23T12:25:14.066 INFO:teuthology.kill:2021-04-23 12:25:14,066.066 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-04-23T12:25:14.068 INFO:teuthology.kill:2021-04-23 12:25:14,068.068 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-04-23T12:25:14.143 INFO:teuthology.kill:2021-04-23 12:25:14,142.142 INFO:teuthology.orchestra.console:Power off smithi140 2021-04-23T12:25:14.155 INFO:teuthology.kill:2021-04-23 12:25:14,155.155 INFO:teuthology.orchestra.console:Power off smithi196 2021-04-23T12:25:22.515 INFO:teuthology.kill:2021-04-23 12:25:22,514.514 INFO:teuthology.orchestra.console:Power off for smithi196 completed 2021-04-23T12:25:26.623 INFO:teuthology.kill:2021-04-23 12:25:26,622.622 INFO:teuthology.orchestra.console:Power off for smithi140 completed 2021-04-23T12:25:26.862 INFO:teuthology.task.internal:roles: ubuntu@smithi140.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-04-23T12:25:26.862 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-04-23T12:25:26.862 INFO:teuthology.misc:Compressing logs... 2021-04-23T12:25:26.862 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-04-23T12:26:26.923 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-04-23T12:26:26.984 INFO:teuthology.kill:No teuthology processes running 2021-04-23T12:26:28.053 INFO:teuthology.kill:Nuking machines: ['smithi140', 'smithi196'] 2021-04-23T12:26:29.005 INFO:teuthology.kill:2021-04-23 12:26:29,004.004 INFO:teuthology.nuke:targets: 2021-04-23T12:26:29.005 INFO:teuthology.kill: smithi140.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJksBhRn5cLwRXEDUwgFWwVoQMukJbOcF2rJNRKszbOlT7DO/r/YwJ8ADFk4rDdnuOsAU5Fou8EY9jkvBZPN9nYZM++dFZy4muky19iNjoVWHJDF/ECssel5KGE1CAutBMIOp5aqZ5SqvM70Qazm4TEZfoFaTq26L7lOqjQ+NywkZ6AUK2TZsy9nm+JubSO7sApmu6IuphMzPjU3AMfNzfOwCzgmhMuG1Ubq8cZqHD8Z4TnXxc3wTtguSvkhkzknQuzxQOhRW/Vk3q5gE6AEyL4gCv0EJwWfCBHDFvj304IsywjWdAmB4kHcJ3bKSIBaXBBvUfL7XpMk0JOQgjLySBglTeDKcHXTHjWwuilD8iWJ+y+pJNdaMlecqQlJ2IbiY8+y5EBPh7dvymjDda2pG0MZsJV2utOQnT4nrL8NQG2Oq/sjhS2rodm/Sscqn6Cw0RL3emjUIK22H6pTAdJUTRccIddbA8M6Yh6WBaoRcQ2aOf2YKJqk+sbJ/LdS5mSS0= 2021-04-23T12:26:29.005 INFO:teuthology.kill: smithi196.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0Xh14rznrNp3onjl7P/NnFSzFW/OdIBw9ufFHtCcqVnCyVtfrMn4oB6m0SAebL4AIePrzSUCzNRvzGBSwjAybKVBuYhO13IUCDlJViYQKxgW77j+BN5ajaQwvkQG9nEDWmOM+UsRiqLTmBZ/Z6zrbP0Qd8MGWe2ePvfnlgMJU+dil27LEz4VwZ2v4T2DtY4nqlf3WbYEBRyKKVW9S1hF4cvkmOJ6NDZauem5pj69xezbSMWMIQLhO5MirqWBZlt8llpntXzzg3KWyMK3RGmzzSRZHXvLIOH8A7wx6a+Z25P4ZKytPxthKjvMr4sEaGgCtnYIz5OWtZIu9lnLvkef7wmSeWQjlRrIQJRR2VaNZ5X+DEWNosnkXbF2Tgq7b97QhC0pmWCVOfI0XYNqjAsqO/GcgoHVzKfVg+v7w323WQFgHViHyhCUPvliLm8emN41VeGfYE49ec64fmOkPXxKTN1IOIOT0YFaHi7alCl4qCx/1Y9QhJRSO7YySXMVF+gM= 2021-04-23T12:26:29.026 INFO:teuthology.kill:2021-04-23 12:26:29,026.026 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-04-23T12:26:29.038 INFO:teuthology.kill:2021-04-23 12:26:29,038.038 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-04-23T12:26:29.105 INFO:teuthology.kill:2021-04-23 12:26:29,105.105 INFO:teuthology.orchestra.console:Power off smithi140 2021-04-23T12:26:29.117 INFO:teuthology.kill:2021-04-23 12:26:29,117.117 INFO:teuthology.orchestra.console:Power off smithi196 2021-04-23T12:26:29.247 INFO:teuthology.kill:2021-04-23 12:26:29,247.247 INFO:teuthology.orchestra.console:Power off for smithi140 completed 2021-04-23T12:26:29.256 INFO:teuthology.kill:2021-04-23 12:26:29,256.256 INFO:teuthology.orchestra.console:Power off for smithi196 completed 2021-04-23T12:26:29.413 INFO:teuthology.kill:2021-04-23 12:26:29,412.412 INFO:teuthology.lock.ops:unlocked smithi140.front.sepia.ceph.com 2021-04-23T12:26:29.432 INFO:teuthology.kill:2021-04-23 12:26:29,431.431 INFO:teuthology.lock.ops:unlocked smithi196.front.sepia.ceph.com 2021-04-23T12:29:14.260 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-04-23T12:29:15.548 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-04-23T12:29:15.549 INFO:teuthology.nuke:Checking targets against current locks 2021-04-23T12:29:15.678 INFO:teuthology.nuke:Not nuking smithi196.front.sepia.ceph.com because description doesn't match 2021-04-23T12:29:15.679 INFO:teuthology.nuke:Not nuking smithi140.front.sepia.ceph.com because description doesn't match