2021-06-01T07:41:24.424 INFO:root:teuthology version: 1.1.0-f359b10d 2021-06-01T07:41:24.515 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi023.front.sepia.ceph.com' reimaging 2021-06-01T07:41:24.516 INFO:teuthology.lock.ops:Updating [ubuntu@smithi023.front.sepia.ceph.com]: reset os type and version on server 2021-06-01T07:41:24.516 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2021-06-01T07:41:24.531 INFO:teuthology.lock.ops:Node 'ubuntu@smithi023.front.sepia.ceph.com' reimaging is complete 2021-06-01T07:41:24.531 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi167.front.sepia.ceph.com' reimaging 2021-06-01T07:41:24.532 INFO:teuthology.lock.ops:Updating [ubuntu@smithi167.front.sepia.ceph.com]: reset os type and version on server 2021-06-01T07:41:24.532 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2021-06-01T07:41:24.547 INFO:teuthology.lock.ops:Node 'ubuntu@smithi167.front.sepia.ceph.com' reimaging is complete 2021-06-01T07:41:24.760 INFO:teuthology.provision.fog.smithi023:Scheduling deploy of centos 8.2 2021-06-01T07:41:24.768 INFO:teuthology.provision.fog.smithi167:Scheduling deploy of centos 8.2 2021-06-01T07:41:25.053 INFO:teuthology.orchestra.console:Power off smithi167 2021-06-01T07:41:25.066 INFO:teuthology.orchestra.console:Power off smithi023 2021-06-01T07:41:33.426 INFO:teuthology.orchestra.console:Power off for smithi023 completed 2021-06-01T07:41:33.527 INFO:teuthology.orchestra.console:Power on smithi023 2021-06-01T07:41:37.533 INFO:teuthology.orchestra.console:Power off for smithi167 completed 2021-06-01T07:41:37.635 INFO:teuthology.orchestra.console:Power on smithi167 2021-06-01T07:41:41.892 INFO:teuthology.orchestra.console:Power on for smithi023 completed 2021-06-01T07:41:41.993 INFO:teuthology.provision.fog.smithi023:Waiting for deploy to finish 2021-06-01T07:41:46.002 INFO:teuthology.orchestra.console:Power on for smithi167 completed 2021-06-01T07:41:46.103 INFO:teuthology.provision.fog.smithi167:Waiting for deploy to finish 2021-06-01T07:46:07.158 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-01T07:46:27.095 INFO:teuthology.provision.fog.smithi167:Node is ready 2021-06-01T07:46:27.114 INFO:teuthology.orchestra.run.smithi167.stdout:smithi167.front.sepia.ceph.com 2021-06-01T07:46:27.170 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.167 smithi167.front.sepia.ceph.com smithi167 2021-06-01T07:46:27.504 INFO:teuthology.provision.fog.smithi167:Deploy complete! 2021-06-01T07:47:11.051 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-01T07:47:11.352 INFO:teuthology.provision.fog.smithi023:Node is ready 2021-06-01T07:47:11.367 INFO:teuthology.orchestra.run.smithi023.stdout:smithi023.front.sepia.ceph.com 2021-06-01T07:47:11.422 INFO:teuthology.orchestra.run.smithi023.stdout:172.21.15.23 smithi023.front.sepia.ceph.com smithi023 2021-06-01T07:47:11.742 INFO:teuthology.provision.fog.smithi023:Deploy complete! 2021-06-01T07:47:11.868 INFO:teuthology.lock.ops:Checking smithi023.front.sepia.ceph.com 2021-06-01T07:47:11.868 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-01T07:47:11.885 INFO:teuthology.lock.ops:Checking smithi167.front.sepia.ceph.com 2021-06-01T07:47:11.885 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-01T07:47:11.900 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: set os type and version on server 2021-06-01T07:47:11.900 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-01T07:47:12.117 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64 2021-06-01T07:47:12.171 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Linux" 2021-06-01T07:47:12.171 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="8 (Core)" 2021-06-01T07:47:12.172 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos" 2021-06-01T07:47:12.172 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora" 2021-06-01T07:47:12.172 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="8" 2021-06-01T07:47:12.172 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el8" 2021-06-01T07:47:12.173 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-06-01T07:47:12.173 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31" 2021-06-01T07:47:12.173 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-06-01T07:47:12.173 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://www.centos.org/" 2021-06-01T07:47:12.174 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-06-01T07:47:12.174 INFO:teuthology.orchestra.run.smithi023.stdout: 2021-06-01T07:47:12.174 INFO:teuthology.orchestra.run.smithi023.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-06-01T07:47:12.174 INFO:teuthology.orchestra.run.smithi023.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-06-01T07:47:12.174 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-06-01T07:47:12.175 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-06-01T07:47:12.175 INFO:teuthology.orchestra.run.smithi023.stdout: 2021-06-01T07:47:12.176 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2021-06-01T07:47:12.191 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: set os type and version on server 2021-06-01T07:47:12.192 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-01T07:47:12.468 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2021-06-01T07:47:12.522 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Linux" 2021-06-01T07:47:12.523 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8 (Core)" 2021-06-01T07:47:12.523 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2021-06-01T07:47:12.523 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2021-06-01T07:47:12.523 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8" 2021-06-01T07:47:12.524 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8" 2021-06-01T07:47:12.524 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-06-01T07:47:12.524 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2021-06-01T07:47:12.524 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-06-01T07:47:12.525 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.centos.org/" 2021-06-01T07:47:12.525 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-06-01T07:47:12.525 INFO:teuthology.orchestra.run.smithi167.stdout: 2021-06-01T07:47:12.525 INFO:teuthology.orchestra.run.smithi167.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-06-01T07:47:12.525 INFO:teuthology.orchestra.run.smithi167.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-06-01T07:47:12.526 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-06-01T07:47:12.526 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-06-01T07:47:12.526 INFO:teuthology.orchestra.run.smithi167.stdout: 2021-06-01T07:47:12.527 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2021-06-01T07:47:12.599 INFO:teuthology.dispatcher.supervisor:Running job 6144703 2021-06-01T07:47:12.600 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-06-01_04:17:03-fs-pacific-distro-basic-smithi/6144703 --name teuthology-2021-06-01_04:17:03-fs-pacific-distro-basic-smithi --description fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-limits} -- /home/teuthworker/archive/teuthology-2021-06-01_04:17:03-fs-pacific-distro-basic-smithi/6144703/orig.config.yaml 2021-06-01T07:47:12.604 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-06-01_04:17:03-fs-pacific-distro-basic-smithi/6144703 2021-06-01T07:47:12.605 INFO:teuthology.dispatcher.supervisor:Job PID: 31281 2021-06-01T07:47:12.605 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-06-01T19:47:59.208 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-06-01T19:47:59.395 INFO:teuthology.kill:Killing Pids: {31281} 2021-06-01T19:48:00.392 INFO:teuthology.kill:Nuking machines: ['smithi023', 'smithi167'] 2021-06-01T19:48:01.183 INFO:teuthology.kill:2021-06-01 19:48:01,183.183 INFO:teuthology.nuke:targets: 2021-06-01T19:48:01.184 INFO:teuthology.kill: smithi023.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFjyMIIFJ/xKVo8opREx8I0L22RMfs4xuQh2UdwFG8ZOrJtDcwJ3pKdUl4Lxc+QFg3N+ZVi6s3BGsvls5ZI9SdVP+ybPU3qmtBp56ZenICpQY3+fBX63uydYplu4W0xeSoH4z2vZ4xPlL0edy/kS2ZUkT4YyYvpXn85Sn9Sy4iB3yLBICR7DcLl/sRjQUr6dMFDD7jc4Di/rPGvFwlhOfNl0gOnWNAhOd5uo6WxUuh5HcJT95XRWsQCNCTwht/GpnV4v6ZF6DS/bgwC2W8urT+/V7xKwZY30ee5HntUZXQWTpEK1aedF8VuBSDzaesip7TV6RpCwbQxzBSevdTKsJRe8luHeA/V/7wPila8qwnZ+mSxzMg4B2T0jEk+hRgY+EMQerS9AMZVwNOUJ7KJip6Ob7EYY3VD7hggalCb8DvhavcTOJQdz7ogAvnHW21cBbaj7vy2wWymAgXXlGFM4qvGo0R8Zs0wODmthIkRbMoXHPDSoPCXlIEtA0d2nbyAKs= 2021-06-01T19:48:01.185 INFO:teuthology.kill: smithi167.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCo73VywaBmXj1JxSOHSzXxgHpleExvrzRDrqJSy+ShIRdHx3Divs3ydPbrsk98+fo9q0D5HIhp4qpgSTWtMvqYuetDC9yWyoNo8b/DzWg1/e+PA9J27wckB1Edsp3ZrlMRyhZ7G2t8PFoAfQDf+zlt1sdMiYSTAHy4jK2eeulDqb9Eur17N2P2CcWoOeL/HehYNqhLn88LRgkeKs/mwPopqq0M5KWt9wzrn+ycGD74N4QcOGUMxHwUAyROsCq8IG6EZx+Vctlynd1VWn/YVE+VjslohoTx3YbnkCcBZ+UjaUH6/X0d/ehGCWYGsSDQu/06Z41Ok3+mARX8U1Xy65AytkWJxgrKkhaOt/2ZEeqorJORhqJA1LZWyc7oZ9da6QwjMyelvpbG/KsQLQMWJWDOGWy12ApAA1ZBfkkrWHXzTf2FnN/XU7HWgSPC64iHg5cbEgDhG+CN+g568VGS3ejm2cnwnCBwDOiVIn1UsmGChxgTmJ8nDI16yDy/KxqiofE= 2021-06-01T19:48:01.185 INFO:teuthology.kill:2021-06-01 19:48:01,183.183 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-01T19:48:01.186 INFO:teuthology.kill:2021-06-01 19:48:01,186.186 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-01T19:48:01.239 INFO:teuthology.kill:2021-06-01 19:48:01,239.239 INFO:teuthology.orchestra.console:Power off smithi023 2021-06-01T19:48:01.251 INFO:teuthology.kill:2021-06-01 19:48:01,251.251 INFO:teuthology.orchestra.console:Power off smithi167 2021-06-01T19:48:09.608 INFO:teuthology.kill:2021-06-01 19:48:09,607.607 INFO:teuthology.orchestra.console:Power off for smithi023 completed 2021-06-01T19:48:09.613 INFO:teuthology.kill:2021-06-01 19:48:09,613.613 INFO:teuthology.orchestra.console:Power off for smithi167 completed 2021-06-01T19:48:09.821 INFO:teuthology.task.internal:roles: ubuntu@smithi023.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2021-06-01T19:48:09.822 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2021-06-01T19:48:09.822 INFO:teuthology.misc:Compressing logs... 2021-06-01T19:48:09.823 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-01T19:48:44.363 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-06-01T19:48:44.424 INFO:teuthology.kill:No teuthology processes running 2021-06-01T19:48:45.369 INFO:teuthology.kill:Nuking machines: ['smithi023', 'smithi167'] 2021-06-01T19:48:46.204 INFO:teuthology.kill:2021-06-01 19:48:46,203.203 INFO:teuthology.nuke:targets: 2021-06-01T19:48:46.204 INFO:teuthology.kill: smithi023.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFjyMIIFJ/xKVo8opREx8I0L22RMfs4xuQh2UdwFG8ZOrJtDcwJ3pKdUl4Lxc+QFg3N+ZVi6s3BGsvls5ZI9SdVP+ybPU3qmtBp56ZenICpQY3+fBX63uydYplu4W0xeSoH4z2vZ4xPlL0edy/kS2ZUkT4YyYvpXn85Sn9Sy4iB3yLBICR7DcLl/sRjQUr6dMFDD7jc4Di/rPGvFwlhOfNl0gOnWNAhOd5uo6WxUuh5HcJT95XRWsQCNCTwht/GpnV4v6ZF6DS/bgwC2W8urT+/V7xKwZY30ee5HntUZXQWTpEK1aedF8VuBSDzaesip7TV6RpCwbQxzBSevdTKsJRe8luHeA/V/7wPila8qwnZ+mSxzMg4B2T0jEk+hRgY+EMQerS9AMZVwNOUJ7KJip6Ob7EYY3VD7hggalCb8DvhavcTOJQdz7ogAvnHW21cBbaj7vy2wWymAgXXlGFM4qvGo0R8Zs0wODmthIkRbMoXHPDSoPCXlIEtA0d2nbyAKs= 2021-06-01T19:48:46.205 INFO:teuthology.kill: smithi167.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCo73VywaBmXj1JxSOHSzXxgHpleExvrzRDrqJSy+ShIRdHx3Divs3ydPbrsk98+fo9q0D5HIhp4qpgSTWtMvqYuetDC9yWyoNo8b/DzWg1/e+PA9J27wckB1Edsp3ZrlMRyhZ7G2t8PFoAfQDf+zlt1sdMiYSTAHy4jK2eeulDqb9Eur17N2P2CcWoOeL/HehYNqhLn88LRgkeKs/mwPopqq0M5KWt9wzrn+ycGD74N4QcOGUMxHwUAyROsCq8IG6EZx+Vctlynd1VWn/YVE+VjslohoTx3YbnkCcBZ+UjaUH6/X0d/ehGCWYGsSDQu/06Z41Ok3+mARX8U1Xy65AytkWJxgrKkhaOt/2ZEeqorJORhqJA1LZWyc7oZ9da6QwjMyelvpbG/KsQLQMWJWDOGWy12ApAA1ZBfkkrWHXzTf2FnN/XU7HWgSPC64iHg5cbEgDhG+CN+g568VGS3ejm2cnwnCBwDOiVIn1UsmGChxgTmJ8nDI16yDy/KxqiofE= 2021-06-01T19:48:46.223 INFO:teuthology.kill:2021-06-01 19:48:46,223.223 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-01T19:48:46.226 INFO:teuthology.kill:2021-06-01 19:48:46,225.225 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-01T19:48:46.278 INFO:teuthology.kill:2021-06-01 19:48:46,277.277 INFO:teuthology.orchestra.console:Power off smithi023 2021-06-01T19:48:46.289 INFO:teuthology.kill:2021-06-01 19:48:46,289.289 INFO:teuthology.orchestra.console:Power off smithi167 2021-06-01T19:48:46.416 INFO:teuthology.kill:2021-06-01 19:48:46,416.416 INFO:teuthology.orchestra.console:Power off for smithi023 completed 2021-06-01T19:48:46.426 INFO:teuthology.kill:2021-06-01 19:48:46,425.425 INFO:teuthology.orchestra.console:Power off for smithi167 completed 2021-06-01T19:48:46.556 INFO:teuthology.kill:2021-06-01 19:48:46,556.556 INFO:teuthology.lock.ops:unlocked smithi023.front.sepia.ceph.com 2021-06-01T19:48:46.562 INFO:teuthology.kill:2021-06-01 19:48:46,562.562 INFO:teuthology.lock.ops:unlocked smithi167.front.sepia.ceph.com 2021-06-01T19:50:46.780 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15