2021-02-09T14:51:13.902 INFO:root:teuthology version: 1.1.0-17c09965 2021-02-09T14:51:13.997 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi031.front.sepia.ceph.com' reimaging 2021-02-09T14:51:13.998 INFO:teuthology.lock.ops:Updating [ubuntu@smithi031.front.sepia.ceph.com]: reset os type and version on server 2021-02-09T14:51:13.998 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2021-02-09T14:51:14.013 INFO:teuthology.lock.ops:Node 'ubuntu@smithi031.front.sepia.ceph.com' reimaging is complete 2021-02-09T14:51:14.013 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi057.front.sepia.ceph.com' reimaging 2021-02-09T14:51:14.013 INFO:teuthology.lock.ops:Updating [ubuntu@smithi057.front.sepia.ceph.com]: reset os type and version on server 2021-02-09T14:51:14.013 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2021-02-09T14:51:14.027 INFO:teuthology.lock.ops:Node 'ubuntu@smithi057.front.sepia.ceph.com' reimaging is complete 2021-02-09T14:51:14.205 INFO:teuthology.provision.fog.smithi031:Scheduling deploy of ubuntu 18.04 2021-02-09T14:51:14.229 INFO:teuthology.provision.fog.smithi057:Scheduling deploy of ubuntu 18.04 2021-02-09T14:51:14.535 INFO:teuthology.orchestra.console:Power off smithi031 2021-02-09T14:51:14.575 INFO:teuthology.orchestra.console:Power off smithi057 2021-02-09T14:51:22.909 INFO:teuthology.orchestra.console:Power off for smithi031 completed 2021-02-09T14:51:23.009 INFO:teuthology.orchestra.console:Power on smithi031 2021-02-09T14:51:27.048 INFO:teuthology.orchestra.console:Power off for smithi057 completed 2021-02-09T14:51:27.149 INFO:teuthology.orchestra.console:Power on smithi057 2021-02-09T14:51:31.381 INFO:teuthology.orchestra.console:Power on for smithi031 completed 2021-02-09T14:51:31.482 INFO:teuthology.provision.fog.smithi031:Waiting for deploy to finish 2021-02-09T14:51:35.511 INFO:teuthology.orchestra.console:Power on for smithi057 completed 2021-02-09T14:51:35.613 INFO:teuthology.provision.fog.smithi057:Waiting for deploy to finish 2021-02-09T14:55:28.465 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-02-09T14:55:34.480 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-02-09T14:55:40.493 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-02-09T14:55:47.531 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-02-09T14:55:53.618 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-09T14:55:54.056 INFO:teuthology.provision.fog.smithi031:Node is ready 2021-02-09T14:55:54.106 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031.front.sepia.ceph.com 2021-02-09T14:55:54.154 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31 smithi031.front.sepia.ceph.com smithi031 2021-02-09T14:55:54.439 INFO:teuthology.provision.fog.smithi031:Deploy complete! 2021-02-09T14:56:02.508 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-02-09T14:56:08.520 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-02-09T14:56:14.532 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-02-09T14:56:20.542 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-02-09T14:56:26.624 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-09T14:56:27.225 INFO:teuthology.provision.fog.smithi057:Node is ready 2021-02-09T14:56:27.231 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057.front.sepia.ceph.com 2021-02-09T14:56:27.283 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57 smithi057.front.sepia.ceph.com smithi057 2021-02-09T14:56:27.573 INFO:teuthology.provision.fog.smithi057:Deploy complete! 2021-02-09T14:56:27.702 INFO:teuthology.lock.ops:Checking smithi031.front.sepia.ceph.com 2021-02-09T14:56:27.703 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-09T14:56:27.718 INFO:teuthology.lock.ops:Checking smithi057.front.sepia.ceph.com 2021-02-09T14:56:27.719 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-09T14:56:27.733 INFO:teuthology.lock.ops:Updating [smithi031.front.sepia.ceph.com]: set os type and version on server 2021-02-09T14:56:27.733 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-09T14:56:28.053 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2021-02-09T14:56:28.101 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="Ubuntu" 2021-02-09T14:56:28.101 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-02-09T14:56:28.101 INFO:teuthology.orchestra.run.smithi031.stdout:ID=ubuntu 2021-02-09T14:56:28.101 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE=debian 2021-02-09T14:56:28.101 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-02-09T14:56:28.101 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="18.04" 2021-02-09T14:56:28.102 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://www.ubuntu.com/" 2021-02-09T14:56:28.102 INFO:teuthology.orchestra.run.smithi031.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-02-09T14:56:28.102 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-02-09T14:56:28.102 INFO:teuthology.orchestra.run.smithi031.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-02-09T14:56:28.102 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_CODENAME=bionic 2021-02-09T14:56:28.102 INFO:teuthology.orchestra.run.smithi031.stdout:UBUNTU_CODENAME=bionic 2021-02-09T14:56:28.103 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2021-02-09T14:56:28.117 INFO:teuthology.lock.ops:Updating [smithi057.front.sepia.ceph.com]: set os type and version on server 2021-02-09T14:56:28.118 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-09T14:56:28.441 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64 2021-02-09T14:56:28.493 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="Ubuntu" 2021-02-09T14:56:28.493 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-02-09T14:56:28.494 INFO:teuthology.orchestra.run.smithi057.stdout:ID=ubuntu 2021-02-09T14:56:28.494 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE=debian 2021-02-09T14:56:28.494 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-02-09T14:56:28.494 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="18.04" 2021-02-09T14:56:28.494 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://www.ubuntu.com/" 2021-02-09T14:56:28.494 INFO:teuthology.orchestra.run.smithi057.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-02-09T14:56:28.494 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-02-09T14:56:28.494 INFO:teuthology.orchestra.run.smithi057.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-02-09T14:56:28.494 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_CODENAME=bionic 2021-02-09T14:56:28.494 INFO:teuthology.orchestra.run.smithi057.stdout:UBUNTU_CODENAME=bionic 2021-02-09T14:56:28.495 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2021-02-09T14:56:28.575 INFO:teuthology.dispatcher.supervisor:Running job 5870994 2021-02-09T14:56:28.576 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/virtualenv/bin/teuthology -v --owner scheduled_trociny@teuthology --archive /home/teuthworker/archive/trociny-2021-02-09_07:56:42-rbd-wip-mgolub-testing-distro-basic-smithi/5870994 --name trociny-2021-02-09_07:56:42-rbd-wip-mgolub-testing-distro-basic-smithi --description rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} -- /home/teuthworker/archive/trociny-2021-02-09_07:56:42-rbd-wip-mgolub-testing-distro-basic-smithi/5870994/orig.config.yaml 2021-02-09T14:56:28.579 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/trociny-2021-02-09_07:56:42-rbd-wip-mgolub-testing-distro-basic-smithi/5870994 2021-02-09T14:56:28.579 INFO:teuthology.dispatcher.supervisor:Job PID: 6219 2021-02-09T14:56:28.579 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-02-10T02:56:45.651 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-02-10T02:56:46.399 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'] 2021-02-10T02:56:46.402 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6'] 2021-02-10T02:56:46.407 INFO:teuthology.misc:Compressing logs... 2021-02-10T02:56:46.418 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-10T02:56:47.731 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-10T02:56:48.962 INFO:teuthology.misc:Compressing logs... 2021-02-10T02:56:49.125 INFO:teuthology.misc:Compressing logs... 2021-02-10T02:57:02.663 INFO:teuthology.orchestra.run.smithi031.stderr:gzip: /var/log/ceph/cluster1-mon.a.log: file size changed while zipping 2021-02-10T02:57:08.945 INFO:teuthology.orchestra.run.smithi031.stderr:gzip: /var/log/ceph/cluster1-mgr.x.log: file size changed while zipping 2021-02-10T03:01:07.855 INFO:teuthology.orchestra.run.smithi031.stderr:gzip: /var/log/ceph/cluster1-osd.1.log: file size changed while zipping 2021-02-10T03:05:35.621 INFO:teuthology.orchestra.run.smithi031.stderr:gzip: /var/log/ceph/cluster1-osd.2.log: file size changed while zipping 2021-02-10T03:09:01.784 INFO:teuthology.orchestra.run.smithi031.stderr:gzip: /var/log/ceph/cluster1-osd.0.log: file size changed while zipping 2021-02-10T03:09:08.976 INFO:teuthology.orchestra.run.smithi031.stderr:gzip: /var/log/ceph/cluster2-mgr.x.log: file size changed while zipping 2021-02-10T03:09:42.390 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/cluster2-osd.2.log: file size changed while zipping 2021-02-10T03:19:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/cluster2-osd.1.log: file size changed while zipping 2021-02-10T03:32:34.834 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/cluster2-osd.0.log: file size changed while zipping 2021-02-10T03:32:51.150 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/cluster2-mon.a.log: file size changed while zipping 2021-02-10T03:42:27.750 INFO:teuthology.orchestra.run.smithi057.stderr:tar: .: file changed as we read it 2021-02-10T03:42:27.964 INFO:teuthology.kill:Killing Pids: {6219} 2021-02-10T03:42:29.089 INFO:teuthology.kill:Nuking machines: ['smithi031', 'smithi057'] 2021-02-10T03:42:29.867 INFO:teuthology.kill:2021-02-10 03:42:29,866.866 INFO:teuthology.nuke:targets: 2021-02-10T03:42:29.867 INFO:teuthology.kill: smithi031.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDORGxRLq7iKEuguB6GpPLcVkQYHlAziPQ1ti3f1nOnQxEd8aJuDp1uvhiC57E/0IYNK2wmv/oJ48Oxd83PVKceXymymn97fwmerM5knFb4YJXOdIrqCjTpgywly+Xgn6ofLgmBsPdr7Ptge5iUYKHEGe+fJwiL/VT2HuyDZ+9wpUod/E29cuHyzb6OTF+ybp08J8ibSf+TxFDjUBRzOkTCyyZ0HXMfqI7GftcqAaGctmEFzWD6hFgXXOGxeilBREsbWpEi8P63fKzKFT6gxDTPy09vN8G/tsnVA4gq62wVvBHr5qCQKqcxtTQwpCQgK2+IXAxJGf6zGFANyRKLCHPt 2021-02-10T03:42:29.867 INFO:teuthology.kill: smithi057.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQClbQwJAPDl7FA3W1OtK3y0B9bCiT3S1KwYnfeKmogYLQGYMHgXJzh+JwZacmi5Uyo9A4fnKQF8OJLPzPhdJI0rWtmjDZ+3w2NKYYyGgR55J8SpvG6Al24OxHK0pFb1MXJugmY9FVO+ZN02G4ko1fqVAAg/TbYWN7mQ7efM9/CzxB4W/jBYxZlqJ549z+F0DmO0PQE0vLYjTbjEp/Jleo4HKeZQCUs2wg9PHqdfRl7UVR5PmLcA12eGzbH9Vgd9YMYc1ohazaRm3POatfJb6fh/ZfYlmz/m4AZDPhIhyJowTN56KG9HLI4gCGwFEbg5vDYAg06pcwsbM+5sSq1zeYWz 2021-02-10T03:42:29.892 INFO:teuthology.kill:2021-02-10 03:42:29,892.892 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-10T03:42:29.895 INFO:teuthology.kill:2021-02-10 03:42:29,895.895 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-10T03:42:29.955 INFO:teuthology.kill:2021-02-10 03:42:29,954.954 INFO:teuthology.orchestra.console:Power off smithi031 2021-02-10T03:42:29.969 INFO:teuthology.kill:2021-02-10 03:42:29,969.969 INFO:teuthology.orchestra.console:Power off smithi057 2021-02-10T03:42:38.406 INFO:teuthology.kill:2021-02-10 03:42:38,405.405 INFO:teuthology.orchestra.console:Power off for smithi057 completed 2021-02-10T03:42:38.552 INFO:teuthology.kill:2021-02-10 03:42:38,552.552 INFO:teuthology.lock.ops:unlocked smithi057.front.sepia.ceph.com 2021-02-10T03:42:42.479 INFO:teuthology.kill:2021-02-10 03:42:42,478.478 INFO:teuthology.orchestra.console:Power off for smithi031 completed 2021-02-10T03:42:42.623 INFO:teuthology.kill:2021-02-10 03:42:42,623.623 INFO:teuthology.lock.ops:unlocked smithi031.front.sepia.ceph.com 2021-02-10T03:44:42.833 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-02-10T03:44:43.075 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-02-10T03:44:43.078 INFO:teuthology.nuke:Checking targets against current locks 2021-02-10T03:44:43.155 INFO:teuthology.nuke:Not nuking smithi057.front.sepia.ceph.com because description doesn't match 2021-02-10T03:44:43.156 INFO:teuthology.nuke:Not nuking smithi031.front.sepia.ceph.com because description doesn't match