2021-06-03T15:51:07.399 INFO:root:teuthology version: 1.1.0-f359b10d 2021-06-03T15:51:07.492 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi018.front.sepia.ceph.com' reimaging 2021-06-03T15:51:07.493 INFO:teuthology.lock.ops:Updating [ubuntu@smithi018.front.sepia.ceph.com]: reset os type and version on server 2021-06-03T15:51:07.494 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2021-06-03T15:51:07.573 INFO:teuthology.lock.ops:Node 'ubuntu@smithi018.front.sepia.ceph.com' reimaging is complete 2021-06-03T15:51:07.574 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi038.front.sepia.ceph.com' reimaging 2021-06-03T15:51:07.574 INFO:teuthology.lock.ops:Updating [ubuntu@smithi038.front.sepia.ceph.com]: reset os type and version on server 2021-06-03T15:51:07.574 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2021-06-03T15:51:07.648 INFO:teuthology.lock.ops:Node 'ubuntu@smithi038.front.sepia.ceph.com' reimaging is complete 2021-06-03T15:51:07.775 INFO:teuthology.provision.fog.smithi018:Scheduling deploy of ubuntu 20.04 2021-06-03T15:51:07.839 INFO:teuthology.provision.fog.smithi038:Scheduling deploy of ubuntu 20.04 2021-06-03T15:51:08.129 INFO:teuthology.orchestra.console:Power off smithi018 2021-06-03T15:51:08.172 INFO:teuthology.orchestra.console:Power off smithi038 2021-06-03T15:51:16.508 INFO:teuthology.orchestra.console:Power off for smithi018 completed 2021-06-03T15:51:16.609 INFO:teuthology.orchestra.console:Power on smithi018 2021-06-03T15:51:20.649 INFO:teuthology.orchestra.console:Power off for smithi038 completed 2021-06-03T15:51:20.750 INFO:teuthology.orchestra.console:Power on smithi038 2021-06-03T15:51:24.977 INFO:teuthology.orchestra.console:Power on for smithi018 completed 2021-06-03T15:51:25.078 INFO:teuthology.provision.fog.smithi018:Waiting for deploy to finish 2021-06-03T15:51:29.111 INFO:teuthology.orchestra.console:Power on for smithi038 completed 2021-06-03T15:51:29.213 INFO:teuthology.provision.fog.smithi038:Waiting for deploy to finish 2021-06-03T15:56:03.026 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-03T15:56:03.499 INFO:teuthology.provision.fog.smithi018:Node is ready 2021-06-03T15:56:03.550 INFO:teuthology.orchestra.run.smithi018.stdout:smithi018.front.sepia.ceph.com 2021-06-03T15:56:03.558 INFO:teuthology.orchestra.run.smithi018.stdout:172.21.15.18 smithi018.front.sepia.ceph.com smithi018 2021-06-03T15:56:03.740 INFO:teuthology.provision.fog.smithi018:Deploy complete! 2021-06-03T15:56:07.064 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-03T15:56:07.531 INFO:teuthology.provision.fog.smithi038:Node is ready 2021-06-03T15:56:07.579 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038.front.sepia.ceph.com 2021-06-03T15:56:07.586 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38 smithi038.front.sepia.ceph.com smithi038 2021-06-03T15:56:07.792 INFO:teuthology.provision.fog.smithi038:Deploy complete! 2021-06-03T15:56:07.948 INFO:teuthology.lock.ops:Checking smithi018.front.sepia.ceph.com 2021-06-03T15:56:07.949 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-03T15:56:07.964 INFO:teuthology.lock.ops:Checking smithi038.front.sepia.ceph.com 2021-06-03T15:56:07.965 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-03T15:56:07.981 INFO:teuthology.lock.ops:Updating [smithi018.front.sepia.ceph.com]: set os type and version on server 2021-06-03T15:56:07.982 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-03T15:56:08.158 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64 2021-06-03T15:56:08.164 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="Ubuntu" 2021-06-03T15:56:08.165 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-06-03T15:56:08.165 INFO:teuthology.orchestra.run.smithi018.stdout:ID=ubuntu 2021-06-03T15:56:08.165 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE=debian 2021-06-03T15:56:08.165 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-06-03T15:56:08.166 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="20.04" 2021-06-03T15:56:08.166 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://www.ubuntu.com/" 2021-06-03T15:56:08.166 INFO:teuthology.orchestra.run.smithi018.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-06-03T15:56:08.166 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-06-03T15:56:08.167 INFO:teuthology.orchestra.run.smithi018.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-06-03T15:56:08.167 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_CODENAME=focal 2021-06-03T15:56:08.167 INFO:teuthology.orchestra.run.smithi018.stdout:UBUNTU_CODENAME=focal 2021-06-03T15:56:08.168 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2021-06-03T15:56:08.184 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: set os type and version on server 2021-06-03T15:56:08.184 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-03T15:56:08.403 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2021-06-03T15:56:08.410 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="Ubuntu" 2021-06-03T15:56:08.410 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-06-03T15:56:08.411 INFO:teuthology.orchestra.run.smithi038.stdout:ID=ubuntu 2021-06-03T15:56:08.411 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE=debian 2021-06-03T15:56:08.411 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-06-03T15:56:08.411 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="20.04" 2021-06-03T15:56:08.412 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://www.ubuntu.com/" 2021-06-03T15:56:08.412 INFO:teuthology.orchestra.run.smithi038.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-06-03T15:56:08.412 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-06-03T15:56:08.412 INFO:teuthology.orchestra.run.smithi038.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-06-03T15:56:08.412 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_CODENAME=focal 2021-06-03T15:56:08.413 INFO:teuthology.orchestra.run.smithi038.stdout:UBUNTU_CODENAME=focal 2021-06-03T15:56:08.413 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2021-06-03T15:56:08.484 INFO:teuthology.dispatcher.supervisor:Running job 6147244 2021-06-03T15:56:08.484 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-06-02_18:29:36-fs-wip-yuri3-testing-2021-06-02-0826-pacific-distro-basic-smithi/6147244 --name yuriw-2021-06-02_18:29:36-fs-wip-yuri3-testing-2021-06-02-0826-pacific-distro-basic-smithi --description fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/fragment} -- /home/teuthworker/archive/yuriw-2021-06-02_18:29:36-fs-wip-yuri3-testing-2021-06-02-0826-pacific-distro-basic-smithi/6147244/orig.config.yaml 2021-06-03T15:56:08.488 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-06-02_18:29:36-fs-wip-yuri3-testing-2021-06-02-0826-pacific-distro-basic-smithi/6147244 2021-06-03T15:56:08.489 INFO:teuthology.dispatcher.supervisor:Job PID: 27451 2021-06-03T15:56:08.489 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-06-04T03:56:58.395 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-06-04T03:56:58.658 INFO:teuthology.kill:Killing Pids: {27451} 2021-06-04T03:57:00.142 INFO:teuthology.kill:Nuking machines: ['smithi018', 'smithi038'] 2021-06-04T03:57:01.296 INFO:teuthology.kill:2021-06-04 03:57:01,295.295 INFO:teuthology.nuke:targets: 2021-06-04T03:57:01.296 INFO:teuthology.kill: smithi018.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD9rUX51we8r67DBkNjHRBf4VNYpdkAGsi6/GS2Bt2Du+9QsWCOozdDlFR1HaFGSoVhUeQywSVn/9/a+ebSS0U5IrdkvrKUeaZ2kmGMAZHGo6WOk0HixuFcdYLxIG+PA4VKUllPA2370wZO1vY6dM2BvAwvOn+KtWlDFFsqo432q/JIe0LwAmMNfXur9/YJe9yA4/48I3BL9MhNA7uStsQf8P7SumlK7IC80jkHPf8ytSpSjmuFBjJt4gP26icYoZJpASfDU5/aui/hkBmMVNFaA4JC40MaS/9w7RrJKv/CkaTz/Yeqh6Ny0CrU588NiBqUDWzk/HEH0qaLl4KIvtshyCddE1UOGNSswL6+A9hMab1AGQuwGqfPpkdxuoiIOlITzR99gOgMZba20UIpCibcX54+uMthmZaC0txDkxAbZFe/gYdJVmYwRsIyZ/MMDTGPOaKFqC74XL+GUv4y+yuO6G/r2FCRg8yVpMCLiZ1b39PXXTGMVIKn5iLs/LzU5E= 2021-06-04T03:57:01.297 INFO:teuthology.kill: smithi038.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFMEelwRDO4T1KwGt5HpDG9T+QiYzYJ8U4Mm+iWdWHshRVDKtivLoR44LgAk2T/4ghPTQFsHZtWCHA7DDSHdyrzCEmQchW4tsLnWNH6rDiomc4VaX1unkgKVBMDZf9RAZU19iR3zPU+L7lUJY1YYb6Ik87b67xj+lVWzqeOcBMXC6LxilHZIE2vmK61wh08j2vM9WozWHfyhWHcF/dzheOxIrPxp3ExmnH1jINLIPE6+C+OHO6p2fFPky1JXy8XDGvNwX+LR8peDUgll6NtA/xXTgv2lfYjQfA2+rtpnP/HgzpLasY8TJIGmCcSZCKPnOhHi+zwjukIQFC4I7AQHypxuTo/rZ0bPgjjoOkyQ2Ges2ZGJ+/JVqYev8B+lD8ZKCJqV6cpejF3RQgA65yE7zGJ/Lyyc7Apk24EUNs9Ofpjy52RJAX8qoHrs42/WybDJ3Mz+okU+Gs3NGzFSxPAHkzWdMZZNY7NOpHk+nfM+HXE6uqLshUL8xtQ+GJ+PaBWuU= 2021-06-04T03:57:01.297 INFO:teuthology.kill:2021-06-04 03:57:01,295.295 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-04T03:57:01.298 INFO:teuthology.kill:2021-06-04 03:57:01,298.298 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-04T03:57:01.376 INFO:teuthology.kill:2021-06-04 03:57:01,376.376 INFO:teuthology.orchestra.console:Power off smithi038 2021-06-04T03:57:01.390 INFO:teuthology.kill:2021-06-04 03:57:01,390.390 INFO:teuthology.orchestra.console:Power off smithi018 2021-06-04T03:57:09.755 INFO:teuthology.kill:2021-06-04 03:57:09,754.754 INFO:teuthology.orchestra.console:Power off for smithi038 completed 2021-06-04T03:57:09.759 INFO:teuthology.kill:2021-06-04 03:57:09,759.759 INFO:teuthology.orchestra.console:Power off for smithi018 completed 2021-06-04T03:57:10.010 INFO:teuthology.task.internal:roles: ubuntu@smithi018.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2021-06-04T03:57:10.011 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2021-06-04T03:57:10.012 INFO:teuthology.misc:Compressing logs... 2021-06-04T03:57:10.014 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-04T03:58:10.076 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-06-04T03:58:10.145 INFO:teuthology.kill:No teuthology processes running 2021-06-04T03:58:11.319 INFO:teuthology.kill:Nuking machines: ['smithi018', 'smithi038'] 2021-06-04T03:58:12.359 INFO:teuthology.kill:2021-06-04 03:58:12,359.359 INFO:teuthology.nuke:targets: 2021-06-04T03:58:12.360 INFO:teuthology.kill: smithi018.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD9rUX51we8r67DBkNjHRBf4VNYpdkAGsi6/GS2Bt2Du+9QsWCOozdDlFR1HaFGSoVhUeQywSVn/9/a+ebSS0U5IrdkvrKUeaZ2kmGMAZHGo6WOk0HixuFcdYLxIG+PA4VKUllPA2370wZO1vY6dM2BvAwvOn+KtWlDFFsqo432q/JIe0LwAmMNfXur9/YJe9yA4/48I3BL9MhNA7uStsQf8P7SumlK7IC80jkHPf8ytSpSjmuFBjJt4gP26icYoZJpASfDU5/aui/hkBmMVNFaA4JC40MaS/9w7RrJKv/CkaTz/Yeqh6Ny0CrU588NiBqUDWzk/HEH0qaLl4KIvtshyCddE1UOGNSswL6+A9hMab1AGQuwGqfPpkdxuoiIOlITzR99gOgMZba20UIpCibcX54+uMthmZaC0txDkxAbZFe/gYdJVmYwRsIyZ/MMDTGPOaKFqC74XL+GUv4y+yuO6G/r2FCRg8yVpMCLiZ1b39PXXTGMVIKn5iLs/LzU5E= 2021-06-04T03:58:12.360 INFO:teuthology.kill: smithi038.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFMEelwRDO4T1KwGt5HpDG9T+QiYzYJ8U4Mm+iWdWHshRVDKtivLoR44LgAk2T/4ghPTQFsHZtWCHA7DDSHdyrzCEmQchW4tsLnWNH6rDiomc4VaX1unkgKVBMDZf9RAZU19iR3zPU+L7lUJY1YYb6Ik87b67xj+lVWzqeOcBMXC6LxilHZIE2vmK61wh08j2vM9WozWHfyhWHcF/dzheOxIrPxp3ExmnH1jINLIPE6+C+OHO6p2fFPky1JXy8XDGvNwX+LR8peDUgll6NtA/xXTgv2lfYjQfA2+rtpnP/HgzpLasY8TJIGmCcSZCKPnOhHi+zwjukIQFC4I7AQHypxuTo/rZ0bPgjjoOkyQ2Ges2ZGJ+/JVqYev8B+lD8ZKCJqV6cpejF3RQgA65yE7zGJ/Lyyc7Apk24EUNs9Ofpjy52RJAX8qoHrs42/WybDJ3Mz+okU+Gs3NGzFSxPAHkzWdMZZNY7NOpHk+nfM+HXE6uqLshUL8xtQ+GJ+PaBWuU= 2021-06-04T03:58:12.379 INFO:teuthology.kill:2021-06-04 03:58:12,379.379 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-04T03:58:12.382 INFO:teuthology.kill:2021-06-04 03:58:12,382.382 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-04T03:58:12.459 INFO:teuthology.kill:2021-06-04 03:58:12,456.456 INFO:teuthology.orchestra.console:Power off smithi018 2021-06-04T03:58:12.486 INFO:teuthology.kill:2021-06-04 03:58:12,486.486 INFO:teuthology.orchestra.console:Power off smithi038 2021-06-04T03:58:12.631 INFO:teuthology.kill:2021-06-04 03:58:12,630.630 INFO:teuthology.orchestra.console:Power off for smithi018 completed 2021-06-04T03:58:12.639 INFO:teuthology.kill:2021-06-04 03:58:12,638.638 INFO:teuthology.orchestra.console:Power off for smithi038 completed 2021-06-04T03:58:12.772 INFO:teuthology.kill:2021-06-04 03:58:12,771.771 INFO:teuthology.lock.ops:unlocked smithi018.front.sepia.ceph.com 2021-06-04T03:58:12.775 INFO:teuthology.kill:2021-06-04 03:58:12,775.775 INFO:teuthology.lock.ops:unlocked smithi038.front.sepia.ceph.com 2021-06-04T04:00:13.095 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-06-04T04:00:13.189 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-06-04T04:00:13.190 INFO:teuthology.nuke:Checking targets against current locks 2021-06-04T04:00:13.265 INFO:teuthology.nuke:Not nuking smithi038.front.sepia.ceph.com because description doesn't match 2021-06-04T04:00:13.266 INFO:teuthology.nuke:Not nuking smithi018.front.sepia.ceph.com because description doesn't match