2021-06-04T22:45:59.691 INFO:root:teuthology version: 1.1.0-f359b10d 2021-06-04T22:45:59.794 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi077.front.sepia.ceph.com' reimaging 2021-06-04T22:45:59.795 INFO:teuthology.lock.ops:Updating [ubuntu@smithi077.front.sepia.ceph.com]: reset os type and version on server 2021-06-04T22:45:59.796 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2021-06-04T22:45:59.809 INFO:teuthology.lock.ops:Node 'ubuntu@smithi077.front.sepia.ceph.com' reimaging is complete 2021-06-04T22:45:59.809 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi097.front.sepia.ceph.com' reimaging 2021-06-04T22:45:59.809 INFO:teuthology.lock.ops:Updating [ubuntu@smithi097.front.sepia.ceph.com]: reset os type and version on server 2021-06-04T22:45:59.810 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2021-06-04T22:45:59.823 INFO:teuthology.lock.ops:Node 'ubuntu@smithi097.front.sepia.ceph.com' reimaging is complete 2021-06-04T22:46:00.028 INFO:teuthology.provision.fog.smithi077:Scheduling deploy of ubuntu 20.04 2021-06-04T22:46:00.032 INFO:teuthology.provision.fog.smithi097:Scheduling deploy of ubuntu 20.04 2021-06-04T22:46:00.464 INFO:teuthology.orchestra.console:Power off smithi077 2021-06-04T22:46:00.478 INFO:teuthology.orchestra.console:Power off smithi097 2021-06-04T22:46:12.955 INFO:teuthology.orchestra.console:Power off for smithi077 completed 2021-06-04T22:46:12.960 INFO:teuthology.orchestra.console:Power off for smithi097 completed 2021-06-04T22:46:13.057 INFO:teuthology.orchestra.console:Power on smithi077 2021-06-04T22:46:13.065 INFO:teuthology.orchestra.console:Power on smithi097 2021-06-04T22:46:17.309 INFO:teuthology.orchestra.console:Power on for smithi077 completed 2021-06-04T22:46:17.314 INFO:teuthology.orchestra.console:Power on for smithi097 completed 2021-06-04T22:46:17.411 INFO:teuthology.provision.fog.smithi077:Waiting for deploy to finish 2021-06-04T22:46:17.417 INFO:teuthology.provision.fog.smithi097:Waiting for deploy to finish 2021-06-04T22:51:18.019 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-04T22:51:18.428 INFO:teuthology.provision.fog.smithi077:Node is ready 2021-06-04T22:51:18.475 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077.front.sepia.ceph.com 2021-06-04T22:51:18.481 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77 smithi077.front.sepia.ceph.com smithi077 2021-06-04T22:51:18.619 INFO:teuthology.provision.fog.smithi077:Deploy complete! 2021-06-04T22:51:19.213 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-04T22:51:19.686 INFO:teuthology.provision.fog.smithi097:Node is ready 2021-06-04T22:51:19.692 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097.front.sepia.ceph.com 2021-06-04T22:51:19.739 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97 smithi097.front.sepia.ceph.com smithi097 2021-06-04T22:51:19.941 INFO:teuthology.provision.fog.smithi097:Deploy complete! 2021-06-04T22:51:20.100 INFO:teuthology.lock.ops:Checking smithi077.front.sepia.ceph.com 2021-06-04T22:51:20.101 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-04T22:51:20.117 INFO:teuthology.lock.ops:Checking smithi097.front.sepia.ceph.com 2021-06-04T22:51:20.117 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-04T22:51:20.131 INFO:teuthology.lock.ops:Updating [smithi077.front.sepia.ceph.com]: set os type and version on server 2021-06-04T22:51:20.131 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-04T22:51:20.303 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2021-06-04T22:51:20.308 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="Ubuntu" 2021-06-04T22:51:20.308 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-06-04T22:51:20.308 INFO:teuthology.orchestra.run.smithi077.stdout:ID=ubuntu 2021-06-04T22:51:20.308 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE=debian 2021-06-04T22:51:20.309 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-06-04T22:51:20.309 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="20.04" 2021-06-04T22:51:20.309 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://www.ubuntu.com/" 2021-06-04T22:51:20.309 INFO:teuthology.orchestra.run.smithi077.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-06-04T22:51:20.310 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-06-04T22:51:20.310 INFO:teuthology.orchestra.run.smithi077.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-06-04T22:51:20.310 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_CODENAME=focal 2021-06-04T22:51:20.310 INFO:teuthology.orchestra.run.smithi077.stdout:UBUNTU_CODENAME=focal 2021-06-04T22:51:20.311 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2021-06-04T22:51:20.326 INFO:teuthology.lock.ops:Updating [smithi097.front.sepia.ceph.com]: set os type and version on server 2021-06-04T22:51:20.326 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-04T22:51:20.538 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2021-06-04T22:51:20.545 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Ubuntu" 2021-06-04T22:51:20.545 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-06-04T22:51:20.545 INFO:teuthology.orchestra.run.smithi097.stdout:ID=ubuntu 2021-06-04T22:51:20.546 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE=debian 2021-06-04T22:51:20.546 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-06-04T22:51:20.546 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="20.04" 2021-06-04T22:51:20.546 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.ubuntu.com/" 2021-06-04T22:51:20.547 INFO:teuthology.orchestra.run.smithi097.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-06-04T22:51:20.547 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-06-04T22:51:20.547 INFO:teuthology.orchestra.run.smithi097.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-06-04T22:51:20.547 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_CODENAME=focal 2021-06-04T22:51:20.548 INFO:teuthology.orchestra.run.smithi097.stdout:UBUNTU_CODENAME=focal 2021-06-04T22:51:20.548 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2021-06-04T22:51:20.621 INFO:teuthology.dispatcher.supervisor:Running job 6152626 2021-06-04T22:51:20.622 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-06-04_18:17:44-fs-wip-yuri4-testing-2021-06-04-0748-pacific-distro-basic-smithi/6152626 --name yuriw-2021-06-04_18:17:44-fs-wip-yuri4-testing-2021-06-04-0748-pacific-distro-basic-smithi --description fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/damage} -- /home/teuthworker/archive/yuriw-2021-06-04_18:17:44-fs-wip-yuri4-testing-2021-06-04-0748-pacific-distro-basic-smithi/6152626/orig.config.yaml 2021-06-04T22:51:20.625 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-06-04_18:17:44-fs-wip-yuri4-testing-2021-06-04-0748-pacific-distro-basic-smithi/6152626 2021-06-04T22:51:20.626 INFO:teuthology.dispatcher.supervisor:Job PID: 19787 2021-06-04T22:51:20.626 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-06-05T10:52:09.455 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-06-05T10:52:09.707 INFO:teuthology.kill:Killing Pids: {19787} 2021-06-05T10:52:10.809 INFO:teuthology.kill:Nuking machines: ['smithi077', 'smithi097'] 2021-06-05T10:52:11.781 INFO:teuthology.kill:2021-06-05 10:52:11,780.780 INFO:teuthology.nuke:targets: 2021-06-05T10:52:11.781 INFO:teuthology.kill: smithi077.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMIJ8Yh11CGDwCy9aQHFWokaRJkoiz9DlII/QztobjYn2T8J6/ita8S8XF585PnTRZHRB3TFcPagT2p6bV3fUNQW/Mr3FkHOsE8bXblTA57KKIVs34N13ODwps67M5etgcwLK1Uik/LJGI8VtIyKZxexDD19s5RalPY5s1l0tVXZWQWQWwxqclqvKlBsAE/aIdNV9qwL/EmtmTMMctG6Vur/P4/gId5jRGWEy7PUOt45CDxBE39kRU+dMWpFf4/cNTYlv7K9Lqa5dOD5COmS9AUFbZoAHewMsyjunZ1TC6m72ThchpTsKkyMAUShgIXImK/ff+1nnH03bxIG/TaNIFJHYF5at4MPuW6raZxi6yzSqM14kfs6MPQGcdErw6+msJ4U00f4ErrgYEYBAl7JCoz37cZ8WjEejqirkbNkbkX0qpzxniw6mieprJ/NgyDOJe9EmBUIJvXcEqHJKAkYS4LzA/AskCCR1CA7jboQQJXMd0OG2l2PVibQilPWwu/FE= 2021-06-05T10:52:11.782 INFO:teuthology.kill: smithi097.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIPZGToHvQlBHDWCJ9m6YwwS278STa3plFmXzniQU1CfL2Y1SLHqSpXqoY7TOY6pUJ0bzUvOAQQ+cB4h3zvnbOKLWcUM2Bt1UwrrigsiM3IJwG3cdu/0CFWpIeVFxsWUdFl16HJAvZ4CeSotqomdtG7/g+FTlzfIaLxYrEZkKSuELbhzzw8Z7LLGqNyFDKFYUpTfBvdcBTLxyv2ybk1+C3lPez1+D5YpT/SJ/lTypnI2MP+tAV9NgAPMsqiYkE48rPf0ms7VejVghJFO4Fwiref+uJ+Sow435pOevy6CUQXDINa/9cUh0ApGYqdK0pv7+uWa7/2gn1/cR0Zgcvbo6fvLvAEPZ+1+6nvikpcuy02zL9MeGPzdjdt4Ag4qPr6+Og91vZjTbbCmAz/gBodPThfuc+TK/TdhUjcZEmXYCN2hdp6lYeC71k2+pwju2CIRSHdBRngt6JCDED3WUZhq5V13q8g+8yA/PpsH6Ry2TFYCUjdM7AOC77UP5b5Vf698s= 2021-06-05T10:52:11.782 INFO:teuthology.kill:2021-06-05 10:52:11,781.781 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-05T10:52:11.783 INFO:teuthology.kill:2021-06-05 10:52:11,783.783 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-05T10:52:11.849 INFO:teuthology.kill:2021-06-05 10:52:11,849.849 INFO:teuthology.orchestra.console:Power off smithi077 2021-06-05T10:52:11.864 INFO:teuthology.kill:2021-06-05 10:52:11,863.863 INFO:teuthology.orchestra.console:Power off smithi097 2021-06-05T10:52:20.216 INFO:teuthology.kill:2021-06-05 10:52:20,215.215 INFO:teuthology.orchestra.console:Power off for smithi077 completed 2021-06-05T10:52:24.346 INFO:teuthology.kill:2021-06-05 10:52:24,345.345 INFO:teuthology.orchestra.console:Power off for smithi097 completed 2021-06-05T10:52:24.639 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2021-06-05T10:52:24.640 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2021-06-05T10:52:24.640 INFO:teuthology.misc:Compressing logs... 2021-06-05T10:52:24.641 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-05T10:53:24.704 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-06-05T10:53:24.775 INFO:teuthology.kill:No teuthology processes running 2021-06-05T10:53:25.820 INFO:teuthology.kill:Nuking machines: ['smithi077', 'smithi097'] 2021-06-05T10:53:26.775 INFO:teuthology.kill:2021-06-05 10:53:26,775.775 INFO:teuthology.nuke:targets: 2021-06-05T10:53:26.776 INFO:teuthology.kill: smithi077.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMIJ8Yh11CGDwCy9aQHFWokaRJkoiz9DlII/QztobjYn2T8J6/ita8S8XF585PnTRZHRB3TFcPagT2p6bV3fUNQW/Mr3FkHOsE8bXblTA57KKIVs34N13ODwps67M5etgcwLK1Uik/LJGI8VtIyKZxexDD19s5RalPY5s1l0tVXZWQWQWwxqclqvKlBsAE/aIdNV9qwL/EmtmTMMctG6Vur/P4/gId5jRGWEy7PUOt45CDxBE39kRU+dMWpFf4/cNTYlv7K9Lqa5dOD5COmS9AUFbZoAHewMsyjunZ1TC6m72ThchpTsKkyMAUShgIXImK/ff+1nnH03bxIG/TaNIFJHYF5at4MPuW6raZxi6yzSqM14kfs6MPQGcdErw6+msJ4U00f4ErrgYEYBAl7JCoz37cZ8WjEejqirkbNkbkX0qpzxniw6mieprJ/NgyDOJe9EmBUIJvXcEqHJKAkYS4LzA/AskCCR1CA7jboQQJXMd0OG2l2PVibQilPWwu/FE= 2021-06-05T10:53:26.777 INFO:teuthology.kill: smithi097.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIPZGToHvQlBHDWCJ9m6YwwS278STa3plFmXzniQU1CfL2Y1SLHqSpXqoY7TOY6pUJ0bzUvOAQQ+cB4h3zvnbOKLWcUM2Bt1UwrrigsiM3IJwG3cdu/0CFWpIeVFxsWUdFl16HJAvZ4CeSotqomdtG7/g+FTlzfIaLxYrEZkKSuELbhzzw8Z7LLGqNyFDKFYUpTfBvdcBTLxyv2ybk1+C3lPez1+D5YpT/SJ/lTypnI2MP+tAV9NgAPMsqiYkE48rPf0ms7VejVghJFO4Fwiref+uJ+Sow435pOevy6CUQXDINa/9cUh0ApGYqdK0pv7+uWa7/2gn1/cR0Zgcvbo6fvLvAEPZ+1+6nvikpcuy02zL9MeGPzdjdt4Ag4qPr6+Og91vZjTbbCmAz/gBodPThfuc+TK/TdhUjcZEmXYCN2hdp6lYeC71k2+pwju2CIRSHdBRngt6JCDED3WUZhq5V13q8g+8yA/PpsH6Ry2TFYCUjdM7AOC77UP5b5Vf698s= 2021-06-05T10:53:26.798 INFO:teuthology.kill:2021-06-05 10:53:26,798.798 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-05T10:53:26.801 INFO:teuthology.kill:2021-06-05 10:53:26,801.801 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-05T10:53:26.864 INFO:teuthology.kill:2021-06-05 10:53:26,864.864 INFO:teuthology.orchestra.console:Power off smithi097 2021-06-05T10:53:26.878 INFO:teuthology.kill:2021-06-05 10:53:26,877.877 INFO:teuthology.orchestra.console:Power off smithi077 2021-06-05T10:53:27.009 INFO:teuthology.kill:2021-06-05 10:53:27,008.008 INFO:teuthology.orchestra.console:Power off for smithi097 completed 2021-06-05T10:53:27.021 INFO:teuthology.kill:2021-06-05 10:53:27,021.021 INFO:teuthology.orchestra.console:Power off for smithi077 completed 2021-06-05T10:53:27.151 INFO:teuthology.kill:2021-06-05 10:53:27,150.150 INFO:teuthology.lock.ops:unlocked smithi097.front.sepia.ceph.com 2021-06-05T10:53:27.160 INFO:teuthology.kill:2021-06-05 10:53:27,160.160 INFO:teuthology.lock.ops:unlocked smithi077.front.sepia.ceph.com 2021-06-05T10:55:27.461 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-06-05T10:55:27.574 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-06-05T10:55:27.574 INFO:teuthology.nuke:Checking targets against current locks 2021-06-05T10:55:27.645 INFO:teuthology.nuke:Not nuking smithi077.front.sepia.ceph.com because description doesn't match 2021-06-05T10:55:27.646 INFO:teuthology.nuke:Not nuking smithi097.front.sepia.ceph.com because description doesn't match