2021-08-10T07:57:41.429 INFO:root:teuthology version: 1.1.0-321319b1 2021-08-10T07:57:41.530 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi006.front.sepia.ceph.com' reimaging 2021-08-10T07:57:41.531 INFO:teuthology.lock.ops:Updating [ubuntu@smithi006.front.sepia.ceph.com]: reset os type and version on server 2021-08-10T07:57:41.531 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2021-08-10T07:57:41.546 INFO:teuthology.lock.ops:Node 'ubuntu@smithi006.front.sepia.ceph.com' reimaging is complete 2021-08-10T07:57:41.546 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi028.front.sepia.ceph.com' reimaging 2021-08-10T07:57:41.547 INFO:teuthology.lock.ops:Updating [ubuntu@smithi028.front.sepia.ceph.com]: reset os type and version on server 2021-08-10T07:57:41.547 INFO:teuthology.lock.ops:Updating smithi028.front.sepia.ceph.com on lock server 2021-08-10T07:57:41.562 INFO:teuthology.lock.ops:Node 'ubuntu@smithi028.front.sepia.ceph.com' reimaging is complete 2021-08-10T07:57:41.752 INFO:teuthology.provision.fog.smithi006:Scheduling deploy of ubuntu 20.04 2021-08-10T07:57:41.773 INFO:teuthology.provision.fog.smithi028:Scheduling deploy of ubuntu 20.04 2021-08-10T07:57:42.085 INFO:teuthology.orchestra.console:Power off smithi006 2021-08-10T07:57:42.127 INFO:teuthology.orchestra.console:Power off smithi028 2021-08-10T07:57:50.462 INFO:teuthology.orchestra.console:Power off for smithi006 completed 2021-08-10T07:57:50.501 INFO:teuthology.orchestra.console:Power off for smithi028 completed 2021-08-10T07:57:50.564 INFO:teuthology.orchestra.console:Power on smithi006 2021-08-10T07:57:50.603 INFO:teuthology.orchestra.console:Power on smithi028 2021-08-10T07:57:58.938 INFO:teuthology.orchestra.console:Power on for smithi006 completed 2021-08-10T07:57:58.963 INFO:teuthology.orchestra.console:Power on for smithi028 completed 2021-08-10T07:57:59.040 INFO:teuthology.provision.fog.smithi006:Waiting for deploy to finish 2021-08-10T07:57:59.064 INFO:teuthology.provision.fog.smithi028:Waiting for deploy to finish 2021-08-10T08:02:27.716 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-08-10T08:02:28.233 INFO:teuthology.provision.fog.smithi028:Node is ready 2021-08-10T08:02:28.240 INFO:teuthology.orchestra.run.smithi028.stdout:smithi028.front.sepia.ceph.com 2021-08-10T08:02:28.247 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28 smithi028.front.sepia.ceph.com smithi028 2021-08-10T08:02:28.468 INFO:teuthology.provision.fog.smithi028:Deploy complete! 2021-08-10T08:02:36.735 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-08-10T08:02:37.201 INFO:teuthology.provision.fog.smithi006:Node is ready 2021-08-10T08:02:37.208 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006.front.sepia.ceph.com 2021-08-10T08:02:37.215 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6 smithi006.front.sepia.ceph.com smithi006 2021-08-10T08:02:37.410 INFO:teuthology.provision.fog.smithi006:Deploy complete! 2021-08-10T08:02:37.642 INFO:teuthology.lock.ops:Checking smithi006.front.sepia.ceph.com 2021-08-10T08:02:37.644 INFO:teuthology.lock.ops:New key found. Updating... 2021-08-10T08:02:37.664 INFO:teuthology.lock.ops:Checking smithi028.front.sepia.ceph.com 2021-08-10T08:02:37.665 INFO:teuthology.lock.ops:New key found. Updating... 2021-08-10T08:02:37.700 INFO:teuthology.lock.ops:Updating [smithi006.front.sepia.ceph.com]: set os type and version on server 2021-08-10T08:02:37.703 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-10T08:02:37.910 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2021-08-10T08:02:37.920 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="Ubuntu" 2021-08-10T08:02:37.921 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-08-10T08:02:37.921 INFO:teuthology.orchestra.run.smithi006.stdout:ID=ubuntu 2021-08-10T08:02:37.921 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE=debian 2021-08-10T08:02:37.921 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-08-10T08:02:37.922 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="20.04" 2021-08-10T08:02:37.922 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://www.ubuntu.com/" 2021-08-10T08:02:37.922 INFO:teuthology.orchestra.run.smithi006.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-08-10T08:02:37.922 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-08-10T08:02:37.923 INFO:teuthology.orchestra.run.smithi006.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-08-10T08:02:37.923 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_CODENAME=focal 2021-08-10T08:02:37.923 INFO:teuthology.orchestra.run.smithi006.stdout:UBUNTU_CODENAME=focal 2021-08-10T08:02:37.924 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2021-08-10T08:02:37.942 INFO:teuthology.lock.ops:Updating [smithi028.front.sepia.ceph.com]: set os type and version on server 2021-08-10T08:02:37.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-10T08:02:38.167 INFO:teuthology.orchestra.run.smithi028.stdout:x86_64 2021-08-10T08:02:38.175 INFO:teuthology.orchestra.run.smithi028.stdout:NAME="Ubuntu" 2021-08-10T08:02:38.175 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-08-10T08:02:38.175 INFO:teuthology.orchestra.run.smithi028.stdout:ID=ubuntu 2021-08-10T08:02:38.175 INFO:teuthology.orchestra.run.smithi028.stdout:ID_LIKE=debian 2021-08-10T08:02:38.176 INFO:teuthology.orchestra.run.smithi028.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-08-10T08:02:38.176 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION_ID="20.04" 2021-08-10T08:02:38.176 INFO:teuthology.orchestra.run.smithi028.stdout:HOME_URL="https://www.ubuntu.com/" 2021-08-10T08:02:38.176 INFO:teuthology.orchestra.run.smithi028.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-08-10T08:02:38.177 INFO:teuthology.orchestra.run.smithi028.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-08-10T08:02:38.177 INFO:teuthology.orchestra.run.smithi028.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-08-10T08:02:38.177 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION_CODENAME=focal 2021-08-10T08:02:38.177 INFO:teuthology.orchestra.run.smithi028.stdout:UBUNTU_CODENAME=focal 2021-08-10T08:02:38.178 INFO:teuthology.lock.ops:Updating smithi028.front.sepia.ceph.com on lock server 2021-08-10T08:02:38.262 INFO:teuthology.dispatcher.supervisor:Running job 6329212 2021-08-10T08:02:38.263 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-08-09_20:42:46-rados-wip-yuri3-testing-2021-08-09-1006-distro-basic-smithi/6329212 --name yuriw-2021-08-09_20:42:46-rados-wip-yuri3-testing-2021-08-09-1006-distro-basic-smithi --description rados/upgrade/parallel/{0-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} -- /home/teuthworker/archive/yuriw-2021-08-09_20:42:46-rados-wip-yuri3-testing-2021-08-09-1006-distro-basic-smithi/6329212/orig.config.yaml 2021-08-10T08:02:38.265 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-08-09_20:42:46-rados-wip-yuri3-testing-2021-08-09-1006-distro-basic-smithi/6329212 2021-08-10T08:02:38.266 INFO:teuthology.dispatcher.supervisor:Job PID: 1184 2021-08-10T08:02:38.266 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-08-10T20:03:31.139 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-08-10T20:03:31.345 INFO:teuthology.kill:Killing Pids: {1184} 2021-08-10T20:03:32.501 INFO:teuthology.kill:Nuking machines: ['smithi006', 'smithi028'] 2021-08-10T20:03:33.538 INFO:teuthology.kill:2021-08-10 20:03:33,538.538 INFO:teuthology.nuke:targets: 2021-08-10T20:03:33.540 INFO:teuthology.kill: smithi006.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWeJkO/pJ4uhJsW+/Q7BJIIGylw+Lr2bLnDvOpxRYI1rzGR3ySF6AilK3f1f79j68sImRfqPTwAHcukfMgUkw4tUScj5aPC/rzbe7F6Gip/pPnHZyQ3vCo3ARwwBm0+NBo0GMerCiAr1YOESsa3gM/zS3SxpIOZOAospiIMFXxAE0YLnLxeFwibnb8upZMwWKf/lYIAN3ye8GSeOsk9dPe62QUH0GRevcwNpBPSL4LOxRK+e+R2zwL8aPTuwkBU+9gb/h8iiK4hLnQ5hR9LKgBgEoHO+zFldyoNoC6a5DyQ6fGpwdYYdj2+T1mbUxPUYay8fhQt5jGFIkIvuHN9qlsbdtM0QufOETg0JV1VU3xYsoupfXluVpjREODEK5muDkFuS39GHPgqcj5Je9kjq1EiHnYKgZNTv1FsCrtL7sSoGR5LdzMJByFlK5W4u7yXeBkOhF1m+72YNUjWCX7SKpiVDwzvtV/hhqh5s7mTVo9z2064xfmhh1Kyt7NTlmJ5J8= 2021-08-10T20:03:33.540 INFO:teuthology.kill: smithi028.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCry/U+9byHO/iNcOiSOO8DdfFEZ5C+kgixjISAnvVa2c0UYWv7GXluUpyNKW4DGoau7l6LIOuyoi0hd3fWGHFyAX445WACGDn0HhfyfHpekijRNP7lgAT0J4uDj9Le2f6gqCCDcwEPR/VdyVolGic7zinE+9z+6F6IbQTISq1ZKoCpLR/dq2M29eW0s4y7cHUy9DubBGFf1FDtQJVPKj/j3TmtK94iezDgUHafqO4I1SNJkuTf5iBL0H3szPBADKtLWgqH22DEugag8PyDQtgouGRZuuaCtGrsZ2Q6Rae3Ohdp2/FqT9lWMw8B2nlmtOhX8RFNrO1QcOf9dirlirQoE36rjNY2kqq9xGRBfwc2PESdlpibBUh0mZjSPYK+3Z6lkHmGsoTxW7h8PQZHqMFL14kUDv5n10Tjcqjk6ZJL4eSMhNrTvKQ+nOZ/J1PKdRAijqe1TYq1CMClsCN/L7EH+Gvuo7wAaRcdi1DWAGeSUnlJexV96kgbkjYEnBP4vB0= 2021-08-10T20:03:33.541 INFO:teuthology.kill:2021-08-10 20:03:33,538.538 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-10T20:03:33.541 INFO:teuthology.kill:2021-08-10 20:03:33,541.541 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-10T20:03:33.601 INFO:teuthology.kill:2021-08-10 20:03:33,601.601 INFO:teuthology.orchestra.console:Power off smithi006 2021-08-10T20:03:33.616 INFO:teuthology.kill:2021-08-10 20:03:33,616.616 INFO:teuthology.orchestra.console:Power off smithi028 2021-08-10T20:03:46.084 INFO:teuthology.kill:2021-08-10 20:03:46,084.084 INFO:teuthology.orchestra.console:Power off for smithi006 completed 2021-08-10T20:03:46.101 INFO:teuthology.kill:2021-08-10 20:03:46,100.100 INFO:teuthology.orchestra.console:Power off for smithi028 completed 2021-08-10T20:03:46.350 INFO:teuthology.task.internal:roles: ubuntu@smithi006.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2021-08-10T20:03:46.350 INFO:teuthology.task.internal:roles: ubuntu@smithi028.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2021-08-10T20:03:46.351 INFO:teuthology.misc:Compressing logs... 2021-08-10T20:03:46.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-08-10T20:04:20.815 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-08-10T20:04:20.893 INFO:teuthology.kill:No teuthology processes running 2021-08-10T20:04:22.143 INFO:teuthology.kill:Nuking machines: ['smithi006', 'smithi028'] 2021-08-10T20:04:23.297 INFO:teuthology.kill:2021-08-10 20:04:23,297.297 INFO:teuthology.nuke:targets: 2021-08-10T20:04:23.298 INFO:teuthology.kill: smithi006.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWeJkO/pJ4uhJsW+/Q7BJIIGylw+Lr2bLnDvOpxRYI1rzGR3ySF6AilK3f1f79j68sImRfqPTwAHcukfMgUkw4tUScj5aPC/rzbe7F6Gip/pPnHZyQ3vCo3ARwwBm0+NBo0GMerCiAr1YOESsa3gM/zS3SxpIOZOAospiIMFXxAE0YLnLxeFwibnb8upZMwWKf/lYIAN3ye8GSeOsk9dPe62QUH0GRevcwNpBPSL4LOxRK+e+R2zwL8aPTuwkBU+9gb/h8iiK4hLnQ5hR9LKgBgEoHO+zFldyoNoC6a5DyQ6fGpwdYYdj2+T1mbUxPUYay8fhQt5jGFIkIvuHN9qlsbdtM0QufOETg0JV1VU3xYsoupfXluVpjREODEK5muDkFuS39GHPgqcj5Je9kjq1EiHnYKgZNTv1FsCrtL7sSoGR5LdzMJByFlK5W4u7yXeBkOhF1m+72YNUjWCX7SKpiVDwzvtV/hhqh5s7mTVo9z2064xfmhh1Kyt7NTlmJ5J8= 2021-08-10T20:04:23.298 INFO:teuthology.kill: smithi028.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCry/U+9byHO/iNcOiSOO8DdfFEZ5C+kgixjISAnvVa2c0UYWv7GXluUpyNKW4DGoau7l6LIOuyoi0hd3fWGHFyAX445WACGDn0HhfyfHpekijRNP7lgAT0J4uDj9Le2f6gqCCDcwEPR/VdyVolGic7zinE+9z+6F6IbQTISq1ZKoCpLR/dq2M29eW0s4y7cHUy9DubBGFf1FDtQJVPKj/j3TmtK94iezDgUHafqO4I1SNJkuTf5iBL0H3szPBADKtLWgqH22DEugag8PyDQtgouGRZuuaCtGrsZ2Q6Rae3Ohdp2/FqT9lWMw8B2nlmtOhX8RFNrO1QcOf9dirlirQoE36rjNY2kqq9xGRBfwc2PESdlpibBUh0mZjSPYK+3Z6lkHmGsoTxW7h8PQZHqMFL14kUDv5n10Tjcqjk6ZJL4eSMhNrTvKQ+nOZ/J1PKdRAijqe1TYq1CMClsCN/L7EH+Gvuo7wAaRcdi1DWAGeSUnlJexV96kgbkjYEnBP4vB0= 2021-08-10T20:04:23.318 INFO:teuthology.kill:2021-08-10 20:04:23,318.318 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-10T20:04:23.323 INFO:teuthology.kill:2021-08-10 20:04:23,323.323 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-08-10T20:04:23.385 INFO:teuthology.kill:2021-08-10 20:04:23,385.385 INFO:teuthology.orchestra.console:Power off smithi028 2021-08-10T20:04:23.401 INFO:teuthology.kill:2021-08-10 20:04:23,401.401 INFO:teuthology.orchestra.console:Power off smithi006 2021-08-10T20:04:23.533 INFO:teuthology.kill:2021-08-10 20:04:23,533.533 INFO:teuthology.orchestra.console:Power off for smithi028 completed 2021-08-10T20:04:23.542 INFO:teuthology.kill:2021-08-10 20:04:23,541.541 INFO:teuthology.orchestra.console:Power off for smithi006 completed 2021-08-10T20:04:23.691 INFO:teuthology.kill:2021-08-10 20:04:23,690.690 INFO:teuthology.lock.ops:unlocked smithi006.front.sepia.ceph.com 2021-08-10T20:04:23.694 INFO:teuthology.kill:2021-08-10 20:04:23,694.694 INFO:teuthology.lock.ops:unlocked smithi028.front.sepia.ceph.com 2021-08-10T20:06:24.042 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-08-10T20:06:24.159 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-08-10T20:06:24.160 INFO:teuthology.nuke:Checking targets against current locks 2021-08-10T20:06:24.259 INFO:teuthology.nuke:Not nuking smithi028.front.sepia.ceph.com because description doesn't match 2021-08-10T20:06:24.259 INFO:teuthology.nuke:Not nuking smithi006.front.sepia.ceph.com because description doesn't match