2021-07-22T18:11:25.778 INFO:root:teuthology version: 1.1.0-b6f3efab 2021-07-22T18:11:25.888 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi096.front.sepia.ceph.com' reimaging 2021-07-22T18:11:25.889 INFO:teuthology.lock.ops:Updating [ubuntu@smithi096.front.sepia.ceph.com]: reset os type and version on server 2021-07-22T18:11:25.889 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2021-07-22T18:11:25.906 INFO:teuthology.lock.ops:Node 'ubuntu@smithi096.front.sepia.ceph.com' reimaging is complete 2021-07-22T18:11:25.906 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi195.front.sepia.ceph.com' reimaging 2021-07-22T18:11:25.907 INFO:teuthology.lock.ops:Updating [ubuntu@smithi195.front.sepia.ceph.com]: reset os type and version on server 2021-07-22T18:11:25.907 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2021-07-22T18:11:25.923 INFO:teuthology.lock.ops:Node 'ubuntu@smithi195.front.sepia.ceph.com' reimaging is complete 2021-07-22T18:11:26.118 INFO:teuthology.provision.fog.smithi096:Scheduling deploy of ubuntu 20.04 2021-07-22T18:11:26.123 INFO:teuthology.provision.fog.smithi195:Scheduling deploy of ubuntu 20.04 2021-07-22T18:11:26.506 INFO:teuthology.orchestra.console:Power off smithi096 2021-07-22T18:11:26.572 INFO:teuthology.orchestra.console:Power off smithi195 2021-07-22T18:11:34.914 INFO:teuthology.orchestra.console:Power off for smithi096 completed 2021-07-22T18:11:34.944 INFO:teuthology.orchestra.console:Power off for smithi195 completed 2021-07-22T18:11:35.016 INFO:teuthology.orchestra.console:Power on smithi096 2021-07-22T18:11:35.047 INFO:teuthology.orchestra.console:Power on smithi195 2021-07-22T18:11:43.401 INFO:teuthology.orchestra.console:Power on for smithi096 completed 2021-07-22T18:11:43.468 INFO:teuthology.orchestra.console:Power on for smithi195 completed 2021-07-22T18:11:43.502 INFO:teuthology.provision.fog.smithi096:Waiting for deploy to finish 2021-07-22T18:11:43.569 INFO:teuthology.provision.fog.smithi195:Waiting for deploy to finish 2021-07-22T18:16:38.444 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-22T18:16:38.528 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-22T18:16:38.903 INFO:teuthology.provision.fog.smithi096:Node is ready 2021-07-22T18:16:38.910 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096.front.sepia.ceph.com 2021-07-22T18:16:38.958 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96 smithi096.front.sepia.ceph.com smithi096 2021-07-22T18:16:39.011 INFO:teuthology.provision.fog.smithi195:Node is ready 2021-07-22T18:16:39.019 INFO:teuthology.orchestra.run.smithi195.stdout:smithi195.front.sepia.ceph.com 2021-07-22T18:16:39.069 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.15.195 smithi195.front.sepia.ceph.com smithi195 2021-07-22T18:16:39.145 INFO:teuthology.provision.fog.smithi096:Deploy complete! 2021-07-22T18:16:39.310 INFO:teuthology.provision.fog.smithi195:Deploy complete! 2021-07-22T18:16:39.479 INFO:teuthology.lock.ops:Checking smithi096.front.sepia.ceph.com 2021-07-22T18:16:39.480 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-22T18:16:39.495 INFO:teuthology.lock.ops:Checking smithi195.front.sepia.ceph.com 2021-07-22T18:16:39.496 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-22T18:16:39.510 INFO:teuthology.lock.ops:Updating [smithi096.front.sepia.ceph.com]: set os type and version on server 2021-07-22T18:16:39.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-22T18:16:39.721 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2021-07-22T18:16:39.728 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="Ubuntu" 2021-07-22T18:16:39.728 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-07-22T18:16:39.729 INFO:teuthology.orchestra.run.smithi096.stdout:ID=ubuntu 2021-07-22T18:16:39.729 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE=debian 2021-07-22T18:16:39.729 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-07-22T18:16:39.729 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="20.04" 2021-07-22T18:16:39.730 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://www.ubuntu.com/" 2021-07-22T18:16:39.730 INFO:teuthology.orchestra.run.smithi096.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-07-22T18:16:39.730 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-07-22T18:16:39.730 INFO:teuthology.orchestra.run.smithi096.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-07-22T18:16:39.731 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_CODENAME=focal 2021-07-22T18:16:39.731 INFO:teuthology.orchestra.run.smithi096.stdout:UBUNTU_CODENAME=focal 2021-07-22T18:16:39.732 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2021-07-22T18:16:39.748 INFO:teuthology.lock.ops:Updating [smithi195.front.sepia.ceph.com]: set os type and version on server 2021-07-22T18:16:39.748 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-22T18:16:39.922 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2021-07-22T18:16:39.928 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="Ubuntu" 2021-07-22T18:16:39.929 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-07-22T18:16:39.929 INFO:teuthology.orchestra.run.smithi195.stdout:ID=ubuntu 2021-07-22T18:16:39.929 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE=debian 2021-07-22T18:16:39.930 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-07-22T18:16:39.930 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="20.04" 2021-07-22T18:16:39.930 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://www.ubuntu.com/" 2021-07-22T18:16:39.931 INFO:teuthology.orchestra.run.smithi195.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-07-22T18:16:39.932 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-07-22T18:16:39.932 INFO:teuthology.orchestra.run.smithi195.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-07-22T18:16:39.932 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_CODENAME=focal 2021-07-22T18:16:39.932 INFO:teuthology.orchestra.run.smithi195.stdout:UBUNTU_CODENAME=focal 2021-07-22T18:16:39.934 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2021-07-22T18:16:40.036 INFO:teuthology.dispatcher.supervisor:Running job 6287298 2021-07-22T18:16:40.037 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/virtualenv/bin/teuthology -v --owner scheduled_nojha@teuthology --archive /home/teuthworker/archive/nojha-2021-07-22_18:00:50-rados-wip-sseshasa-testing-2021-07-14-1320-distro-basic-smithi/6287298 --name nojha-2021-07-22_18:00:50-rados-wip-sseshasa-testing-2021-07-14-1320-distro-basic-smithi --description rados/monthrash/{ceph clusters/9-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/snaps-few-objects} -- /home/teuthworker/archive/nojha-2021-07-22_18:00:50-rados-wip-sseshasa-testing-2021-07-14-1320-distro-basic-smithi/6287298/orig.config.yaml 2021-07-22T18:16:40.046 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nojha-2021-07-22_18:00:50-rados-wip-sseshasa-testing-2021-07-14-1320-distro-basic-smithi/6287298 2021-07-22T18:16:40.047 INFO:teuthology.dispatcher.supervisor:Job PID: 1050 2021-07-22T18:16:40.048 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-07-23T06:17:29.685 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-07-23T06:17:29.852 INFO:teuthology.kill:Killing Pids: {1050} 2021-07-23T06:17:31.009 INFO:teuthology.kill:Nuking machines: ['smithi096', 'smithi195'] 2021-07-23T06:17:31.934 INFO:teuthology.kill:2021-07-23 06:17:31,934.934 INFO:teuthology.nuke:targets: 2021-07-23T06:17:31.935 INFO:teuthology.kill: smithi096.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFq6oHgPQZYRssLGYg53YLXwtjcAVbwU390VjgzRqh9U9z6I3/ZnX+6q5k4wziEmvuFpLkd8YohMbvHuv+f6/OO3z49i0QpBZnZ7ZDHyol7s4h7mjyojJd1MQwCIdVEDQMosj4f/5YSHkTas7h4pO5Ucl82kZG+0AodDl8EgfxzQby8gp4FU6MB6f734D//QS3UWOdYSZ/9Bhuiw2grVZQrpzcJ57AqdzBtXiEa21D2SJmrRTSvWHagNxnMYzB0tx99hetxV+eJdzPJ+CXd6lASUAWUB/Ng99Bh/NtE4B3fcEcgf5LaweU4Ie84FKhPKSsU9UL9L15X9ee1u5p0Lnzzc7jBtj6KHMmcgryT0/CcJQQCjZflzpww95qlfzQ1YRrLBcvSuDa7RjPd2umDgLQPaX8V+b3ZFwpXCt8lWB9KLVIdX2s6UKIzmFG13+pCJFRaXa7KhpZfnnJTaGBiH0LKkPbNnIiKOWFpVS3Y5vETcCROiW/cDv7+0rqp1aYEsE= 2021-07-23T06:17:31.936 INFO:teuthology.kill: smithi195.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDETaXJlL2bDTyQKYgDqJK6z6xa+whnZfZR7T8WOk4oJ+EClX9v4dXlUtmd/HuDD6lNATk3EC/2v691mbozB38HvYDl3RCRT6gyMXpKq+5u0itsG5hn0ZPuqCedJAtEu5wRXmcNKQ7NOC/jcme35zIUq0s76WLL2EZINUMJijgPHFsQlI+aAgap7cELONsPH5k0sc8k8saPsH2P2+rQnmtjvT8x68+Xi7TqnyrXV86hMCM4s9xdSvWGbVklN1F0OQx+k5gPq1E8Bxxva1EmjaTars7Zaw49zeWoQ5QB9tSbvV9/YLCJvnlAKgnDeYKv4kyPDXM0gjEUz1WBte8mQqrxly9FgUHd811Sgi3JJCxU3EDp0JVbCz31w1cmufLrGaroSuYA/pKUg6IPG2Mx3pfJ/JVhINn6jH+LrmTXiuxqV2MClks+YsdMDeDhM48yl3ixCVzGbH+tA+P9OrKQz7Fe7Hhh84Y52V6Xhw2uxH3b7c7Zt/C5o0dX+54xSvr34M0= 2021-07-23T06:17:31.936 INFO:teuthology.kill:2021-07-23 06:17:31,934.934 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-23T06:17:31.937 INFO:teuthology.kill:2021-07-23 06:17:31,937.937 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-23T06:17:31.996 INFO:teuthology.kill:2021-07-23 06:17:31,996.996 INFO:teuthology.orchestra.console:Power off smithi096 2021-07-23T06:17:32.013 INFO:teuthology.kill:2021-07-23 06:17:32,012.012 INFO:teuthology.orchestra.console:Power off smithi195 2021-07-23T06:17:40.381 INFO:teuthology.kill:2021-07-23 06:17:40,381.381 INFO:teuthology.orchestra.console:Power off for smithi195 completed 2021-07-23T06:17:44.484 INFO:teuthology.kill:2021-07-23 06:17:44,484.484 INFO:teuthology.orchestra.console:Power off for smithi096 completed 2021-07-23T06:17:44.721 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'] 2021-07-23T06:17:44.722 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0'] 2021-07-23T06:17:44.722 INFO:teuthology.misc:Compressing logs... 2021-07-23T06:17:44.723 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-23T06:18:03.083 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b6f3efab84b70e18e86ec271a07564aeb4ed3bac/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-07-23T06:18:03.139 INFO:teuthology.kill:No teuthology processes running 2021-07-23T06:18:04.194 INFO:teuthology.kill:Nuking machines: ['smithi096', 'smithi195'] 2021-07-23T06:18:05.089 INFO:teuthology.kill:2021-07-23 06:18:05,088.088 INFO:teuthology.nuke:targets: 2021-07-23T06:18:05.090 INFO:teuthology.kill: smithi096.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFq6oHgPQZYRssLGYg53YLXwtjcAVbwU390VjgzRqh9U9z6I3/ZnX+6q5k4wziEmvuFpLkd8YohMbvHuv+f6/OO3z49i0QpBZnZ7ZDHyol7s4h7mjyojJd1MQwCIdVEDQMosj4f/5YSHkTas7h4pO5Ucl82kZG+0AodDl8EgfxzQby8gp4FU6MB6f734D//QS3UWOdYSZ/9Bhuiw2grVZQrpzcJ57AqdzBtXiEa21D2SJmrRTSvWHagNxnMYzB0tx99hetxV+eJdzPJ+CXd6lASUAWUB/Ng99Bh/NtE4B3fcEcgf5LaweU4Ie84FKhPKSsU9UL9L15X9ee1u5p0Lnzzc7jBtj6KHMmcgryT0/CcJQQCjZflzpww95qlfzQ1YRrLBcvSuDa7RjPd2umDgLQPaX8V+b3ZFwpXCt8lWB9KLVIdX2s6UKIzmFG13+pCJFRaXa7KhpZfnnJTaGBiH0LKkPbNnIiKOWFpVS3Y5vETcCROiW/cDv7+0rqp1aYEsE= 2021-07-23T06:18:05.090 INFO:teuthology.kill: smithi195.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDETaXJlL2bDTyQKYgDqJK6z6xa+whnZfZR7T8WOk4oJ+EClX9v4dXlUtmd/HuDD6lNATk3EC/2v691mbozB38HvYDl3RCRT6gyMXpKq+5u0itsG5hn0ZPuqCedJAtEu5wRXmcNKQ7NOC/jcme35zIUq0s76WLL2EZINUMJijgPHFsQlI+aAgap7cELONsPH5k0sc8k8saPsH2P2+rQnmtjvT8x68+Xi7TqnyrXV86hMCM4s9xdSvWGbVklN1F0OQx+k5gPq1E8Bxxva1EmjaTars7Zaw49zeWoQ5QB9tSbvV9/YLCJvnlAKgnDeYKv4kyPDXM0gjEUz1WBte8mQqrxly9FgUHd811Sgi3JJCxU3EDp0JVbCz31w1cmufLrGaroSuYA/pKUg6IPG2Mx3pfJ/JVhINn6jH+LrmTXiuxqV2MClks+YsdMDeDhM48yl3ixCVzGbH+tA+P9OrKQz7Fe7Hhh84Y52V6Xhw2uxH3b7c7Zt/C5o0dX+54xSvr34M0= 2021-07-23T06:18:05.109 INFO:teuthology.kill:2021-07-23 06:18:05,108.108 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-23T06:18:05.111 INFO:teuthology.kill:2021-07-23 06:18:05,111.111 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-23T06:18:05.164 INFO:teuthology.kill:2021-07-23 06:18:05,164.164 INFO:teuthology.orchestra.console:Power off smithi195 2021-07-23T06:18:05.178 INFO:teuthology.kill:2021-07-23 06:18:05,178.178 INFO:teuthology.orchestra.console:Power off smithi096 2021-07-23T06:18:05.305 INFO:teuthology.kill:2021-07-23 06:18:05,304.304 INFO:teuthology.orchestra.console:Power off for smithi195 completed 2021-07-23T06:18:05.320 INFO:teuthology.kill:2021-07-23 06:18:05,320.320 INFO:teuthology.orchestra.console:Power off for smithi096 completed 2021-07-23T06:18:05.448 INFO:teuthology.kill:2021-07-23 06:18:05,447.447 INFO:teuthology.lock.ops:unlocked smithi195.front.sepia.ceph.com 2021-07-23T06:18:05.462 INFO:teuthology.kill:2021-07-23 06:18:05,462.462 INFO:teuthology.lock.ops:unlocked smithi096.front.sepia.ceph.com 2021-07-23T06:20:05.734 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-07-23T06:20:05.820 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-07-23T06:20:05.820 INFO:teuthology.nuke:Checking targets against current locks 2021-07-23T06:20:05.891 INFO:teuthology.nuke:Not nuking smithi096.front.sepia.ceph.com because description doesn't match 2021-07-23T06:20:05.891 INFO:teuthology.nuke:Not nuking smithi195.front.sepia.ceph.com because description doesn't match