2021-08-12T09:07:44.994 INFO:root:teuthology version: 1.1.0-321319b1 2021-08-12T09:07:45.107 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi107.front.sepia.ceph.com' reimaging 2021-08-12T09:07:45.108 INFO:teuthology.lock.ops:Updating [ubuntu@smithi107.front.sepia.ceph.com]: reset os type and version on server 2021-08-12T09:07:45.109 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2021-08-12T09:07:45.133 INFO:teuthology.lock.ops:Node 'ubuntu@smithi107.front.sepia.ceph.com' reimaging is complete 2021-08-12T09:07:45.133 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi115.front.sepia.ceph.com' reimaging 2021-08-12T09:07:45.134 INFO:teuthology.lock.ops:Updating [ubuntu@smithi115.front.sepia.ceph.com]: reset os type and version on server 2021-08-12T09:07:45.135 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2021-08-12T09:07:45.182 INFO:teuthology.lock.ops:Node 'ubuntu@smithi115.front.sepia.ceph.com' reimaging is complete 2021-08-12T09:07:45.358 INFO:teuthology.provision.fog.smithi107:Scheduling deploy of ubuntu 20.04 2021-08-12T09:07:45.398 INFO:teuthology.provision.fog.smithi115:Scheduling deploy of ubuntu 20.04 2021-08-12T09:07:45.720 INFO:teuthology.orchestra.console:Power off smithi107 2021-08-12T09:07:45.757 INFO:teuthology.orchestra.console:Power off smithi115 2021-08-12T09:07:54.091 INFO:teuthology.orchestra.console:Power off for smithi107 completed 2021-08-12T09:07:54.124 INFO:teuthology.orchestra.console:Power off for smithi115 completed 2021-08-12T09:07:54.193 INFO:teuthology.orchestra.console:Power on smithi107 2021-08-12T09:07:54.227 INFO:teuthology.orchestra.console:Power on smithi115 2021-08-12T09:08:02.567 INFO:teuthology.orchestra.console:Power on for smithi107 completed 2021-08-12T09:08:02.587 INFO:teuthology.orchestra.console:Power on for smithi115 completed 2021-08-12T09:08:02.669 INFO:teuthology.provision.fog.smithi107:Waiting for deploy to finish 2021-08-12T09:08:02.688 INFO:teuthology.provision.fog.smithi115:Waiting for deploy to finish 2021-08-12T09:12:40.832 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-08-12T09:12:42.764 INFO:teuthology.provision.fog.smithi115:Node is ready 2021-08-12T09:12:42.771 INFO:teuthology.orchestra.run.smithi115.stdout:smithi115.front.sepia.ceph.com 2021-08-12T09:12:42.777 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115 smithi115.front.sepia.ceph.com smithi115 2021-08-12T09:12:43.134 INFO:teuthology.provision.fog.smithi115:Deploy complete! 2021-08-12T09:13:04.772 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-08-12T09:13:05.300 INFO:teuthology.provision.fog.smithi107:Node is ready 2021-08-12T09:13:05.307 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107.front.sepia.ceph.com 2021-08-12T09:13:05.315 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107 smithi107.front.sepia.ceph.com smithi107 2021-08-12T09:13:05.493 INFO:teuthology.provision.fog.smithi107:Deploy complete! 2021-08-12T09:13:05.664 INFO:teuthology.lock.ops:Checking smithi107.front.sepia.ceph.com 2021-08-12T09:13:05.665 INFO:teuthology.lock.ops:Checking smithi115.front.sepia.ceph.com 2021-08-12T09:13:05.667 INFO:teuthology.lock.ops:Updating [smithi107.front.sepia.ceph.com]: set os type and version on server 2021-08-12T09:13:05.667 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-12T09:13:05.846 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2021-08-12T09:13:05.895 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="Ubuntu" 2021-08-12T09:13:05.895 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-08-12T09:13:05.896 INFO:teuthology.orchestra.run.smithi107.stdout:ID=ubuntu 2021-08-12T09:13:05.896 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE=debian 2021-08-12T09:13:05.896 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-08-12T09:13:05.896 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="20.04" 2021-08-12T09:13:05.896 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://www.ubuntu.com/" 2021-08-12T09:13:05.897 INFO:teuthology.orchestra.run.smithi107.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-08-12T09:13:05.897 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-08-12T09:13:05.897 INFO:teuthology.orchestra.run.smithi107.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-08-12T09:13:05.897 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_CODENAME=focal 2021-08-12T09:13:05.898 INFO:teuthology.orchestra.run.smithi107.stdout:UBUNTU_CODENAME=focal 2021-08-12T09:13:05.899 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2021-08-12T09:13:05.916 INFO:teuthology.lock.ops:Updating [smithi115.front.sepia.ceph.com]: set os type and version on server 2021-08-12T09:13:05.917 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-12T09:13:06.170 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2021-08-12T09:13:06.217 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="Ubuntu" 2021-08-12T09:13:06.218 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-08-12T09:13:06.218 INFO:teuthology.orchestra.run.smithi115.stdout:ID=ubuntu 2021-08-12T09:13:06.218 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE=debian 2021-08-12T09:13:06.218 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-08-12T09:13:06.218 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="20.04" 2021-08-12T09:13:06.219 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://www.ubuntu.com/" 2021-08-12T09:13:06.219 INFO:teuthology.orchestra.run.smithi115.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-08-12T09:13:06.219 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-08-12T09:13:06.219 INFO:teuthology.orchestra.run.smithi115.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-08-12T09:13:06.220 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_CODENAME=focal 2021-08-12T09:13:06.220 INFO:teuthology.orchestra.run.smithi115.stdout:UBUNTU_CODENAME=focal 2021-08-12T09:13:06.220 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2021-08-12T09:13:06.301 INFO:teuthology.dispatcher.supervisor:Running job 6333297 2021-08-12T09:13:06.301 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2021-08-10_23:16:44-fs-wip-pdonnell-testing-20210810.201536-distro-basic-smithi/6333297 --name pdonnell-2021-08-10_23:16:44-fs-wip-pdonnell-testing-20210810.201536-distro-basic-smithi --description fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/fragment} -- /home/teuthworker/archive/pdonnell-2021-08-10_23:16:44-fs-wip-pdonnell-testing-20210810.201536-distro-basic-smithi/6333297/orig.config.yaml 2021-08-12T09:13:06.305 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2021-08-10_23:16:44-fs-wip-pdonnell-testing-20210810.201536-distro-basic-smithi/6333297 2021-08-12T09:13:06.306 INFO:teuthology.dispatcher.supervisor:Job PID: 23233 2021-08-12T09:13:06.307 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-08-12T21:13:58.147 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-08-12T21:13:58.377 INFO:teuthology.kill:Killing Pids: {23233} 2021-08-12T21:13:59.605 INFO:teuthology.kill:Nuking machines: ['smithi107', 'smithi115'] 2021-08-12T21:14:00.587 INFO:teuthology.kill:2021-08-12 21:14:00,587.587 INFO:teuthology.nuke:targets: 2021-08-12T21:14:00.588 INFO:teuthology.kill: smithi107.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEfFEHyVTOsL3/uQc/mQ1OiGJVDGDBYihyRs0AK72xrGgEnrZ0w1qVLxOy3oqR7ZrNcv4B0qhXsB2OrQ9xsrecQnfAoc+DZgFos8InFn1sOyZzyJ0hZMcrKC9+2dTC/oJec5FfL0l6N/XsgyZMnpXR6K84NJ4ewIIZ7PKP9Ywpwmaka8txYmQoo8Eyquh1NqLRTf521cozcuzvC71r6hTXAjnmvWMVtoSty2N1AeMLzS2l62biQ79L3/7J+UJr9r35ndUFLixGBdaJH4YONLVMxZuuX1rZBlRurGBGNDqy4sRyGLcLFlmcqt8KebXVXTGc60GLyUkdvNrXyPRuGnbxpb6Xf4yluc5gZJgRkk/8wvedxpTA93rADpqm+a5m9S4WOjSXs5vfhE1Zry7Pp3nfNhd3RGqYKMWS0tCoaRhqxdFPv1/tJ0YbF5RmRI50Mdsv84iV+4sjh3DFs55L7vVzSVZAE0Vmw0g5oRC6DSaH4udQBif2IKq3fpvqMDzFADE= 2021-08-12T21:14:00.589 INFO:teuthology.kill: smithi115.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEfFEHyVTOsL3/uQc/mQ1OiGJVDGDBYihyRs0AK72xrGgEnrZ0w1qVLxOy3oqR7ZrNcv4B0qhXsB2OrQ9xsrecQnfAoc+DZgFos8InFn1sOyZzyJ0hZMcrKC9+2dTC/oJec5FfL0l6N/XsgyZMnpXR6K84NJ4ewIIZ7PKP9Ywpwmaka8txYmQoo8Eyquh1NqLRTf521cozcuzvC71r6hTXAjnmvWMVtoSty2N1AeMLzS2l62biQ79L3/7J+UJr9r35ndUFLixGBdaJH4YONLVMxZuuX1rZBlRurGBGNDqy4sRyGLcLFlmcqt8KebXVXTGc60GLyUkdvNrXyPRuGnbxpb6Xf4yluc5gZJgRkk/8wvedxpTA93rADpqm+a5m9S4WOjSXs5vfhE1Zry7Pp3nfNhd3RGqYKMWS0tCoaRhqxdFPv1/tJ0YbF5RmRI50Mdsv84iV+4sjh3DFs55L7vVzSVZAE0Vmw0g5oRC6DSaH4udQBif2IKq3fpvqMDzFADE= 2021-08-12T21:14:00.589 INFO:teuthology.kill:2021-08-12 21:14:00,587.587 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-12T21:14:00.590 INFO:teuthology.kill:2021-08-12 21:14:00,590.590 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-12T21:14:00.657 INFO:teuthology.kill:2021-08-12 21:14:00,656.656 INFO:teuthology.orchestra.console:Power off smithi107 2021-08-12T21:14:00.671 INFO:teuthology.kill:2021-08-12 21:14:00,670.670 INFO:teuthology.orchestra.console:Power off smithi115 2021-08-12T21:14:13.142 INFO:teuthology.kill:2021-08-12 21:14:13,141.141 INFO:teuthology.orchestra.console:Power off for smithi107 completed 2021-08-12T21:14:13.148 INFO:teuthology.kill:2021-08-12 21:14:13,148.148 INFO:teuthology.orchestra.console:Power off for smithi115 completed 2021-08-12T21:14:13.404 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2021-08-12T21:14:13.404 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2021-08-12T21:14:13.405 INFO:teuthology.misc:Compressing logs... 2021-08-12T21:14:13.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-12T21:14:31.762 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-08-12T21:14:31.826 INFO:teuthology.kill:No teuthology processes running 2021-08-12T21:14:33.093 INFO:teuthology.kill:Nuking machines: ['smithi107', 'smithi115'] 2021-08-12T21:14:34.105 INFO:teuthology.kill:2021-08-12 21:14:34,105.105 INFO:teuthology.nuke:targets: 2021-08-12T21:14:34.106 INFO:teuthology.kill: smithi107.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEfFEHyVTOsL3/uQc/mQ1OiGJVDGDBYihyRs0AK72xrGgEnrZ0w1qVLxOy3oqR7ZrNcv4B0qhXsB2OrQ9xsrecQnfAoc+DZgFos8InFn1sOyZzyJ0hZMcrKC9+2dTC/oJec5FfL0l6N/XsgyZMnpXR6K84NJ4ewIIZ7PKP9Ywpwmaka8txYmQoo8Eyquh1NqLRTf521cozcuzvC71r6hTXAjnmvWMVtoSty2N1AeMLzS2l62biQ79L3/7J+UJr9r35ndUFLixGBdaJH4YONLVMxZuuX1rZBlRurGBGNDqy4sRyGLcLFlmcqt8KebXVXTGc60GLyUkdvNrXyPRuGnbxpb6Xf4yluc5gZJgRkk/8wvedxpTA93rADpqm+a5m9S4WOjSXs5vfhE1Zry7Pp3nfNhd3RGqYKMWS0tCoaRhqxdFPv1/tJ0YbF5RmRI50Mdsv84iV+4sjh3DFs55L7vVzSVZAE0Vmw0g5oRC6DSaH4udQBif2IKq3fpvqMDzFADE= 2021-08-12T21:14:34.107 INFO:teuthology.kill: smithi115.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEfFEHyVTOsL3/uQc/mQ1OiGJVDGDBYihyRs0AK72xrGgEnrZ0w1qVLxOy3oqR7ZrNcv4B0qhXsB2OrQ9xsrecQnfAoc+DZgFos8InFn1sOyZzyJ0hZMcrKC9+2dTC/oJec5FfL0l6N/XsgyZMnpXR6K84NJ4ewIIZ7PKP9Ywpwmaka8txYmQoo8Eyquh1NqLRTf521cozcuzvC71r6hTXAjnmvWMVtoSty2N1AeMLzS2l62biQ79L3/7J+UJr9r35ndUFLixGBdaJH4YONLVMxZuuX1rZBlRurGBGNDqy4sRyGLcLFlmcqt8KebXVXTGc60GLyUkdvNrXyPRuGnbxpb6Xf4yluc5gZJgRkk/8wvedxpTA93rADpqm+a5m9S4WOjSXs5vfhE1Zry7Pp3nfNhd3RGqYKMWS0tCoaRhqxdFPv1/tJ0YbF5RmRI50Mdsv84iV+4sjh3DFs55L7vVzSVZAE0Vmw0g5oRC6DSaH4udQBif2IKq3fpvqMDzFADE= 2021-08-12T21:14:34.126 INFO:teuthology.kill:2021-08-12 21:14:34,125.125 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-12T21:14:34.129 INFO:teuthology.kill:2021-08-12 21:14:34,128.128 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-12T21:14:34.185 INFO:teuthology.kill:2021-08-12 21:14:34,185.185 INFO:teuthology.orchestra.console:Power off smithi107 2021-08-12T21:14:34.199 INFO:teuthology.kill:2021-08-12 21:14:34,198.198 INFO:teuthology.orchestra.console:Power off smithi115 2021-08-12T21:14:34.327 INFO:teuthology.kill:2021-08-12 21:14:34,327.327 INFO:teuthology.orchestra.console:Power off for smithi107 completed 2021-08-12T21:14:34.340 INFO:teuthology.kill:2021-08-12 21:14:34,340.340 INFO:teuthology.orchestra.console:Power off for smithi115 completed 2021-08-12T21:14:34.469 INFO:teuthology.kill:2021-08-12 21:14:34,468.468 INFO:teuthology.lock.ops:unlocked smithi107.front.sepia.ceph.com 2021-08-12T21:14:34.480 INFO:teuthology.kill:2021-08-12 21:14:34,480.480 INFO:teuthology.lock.ops:unlocked smithi115.front.sepia.ceph.com 2021-08-12T21:16:34.777 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-08-12T21:16:34.872 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-08-12T21:16:34.872 INFO:teuthology.nuke:Checking targets against current locks 2021-08-12T21:16:34.988 INFO:teuthology.nuke:Not nuking smithi115.front.sepia.ceph.com because description doesn't match 2021-08-12T21:16:34.989 INFO:teuthology.nuke:Not nuking smithi107.front.sepia.ceph.com because description doesn't match