2020-11-23T13:49:38.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:49:47.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:49:49.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:49:56.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:49:58.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:50:05.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:50:07.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:50:14.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:50:16.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:50:23.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:50:25.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:50:32.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:50:34.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:50:44.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:50:55.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:50:57.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:51:06.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:51:06.622 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-23T13:51:12.629 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:51:12.640 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-23T13:51:18.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:51:18.653 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-23T13:51:19.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:51:24.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:51:24.666 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-23T13:51:28.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:51:30.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:51:31.735 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-23T13:51:34.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:51:34.724 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-23T13:51:37.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:51:37.902 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-23T13:51:37.903 INFO:teuthology.orchestra.run.smithi086:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-23T13:51:38.285 INFO:teuthology.provision.fog.smithi086:Node is ready 2020-11-23T13:51:38.285 INFO:teuthology.orchestra.run.smithi086:> hostname 2020-11-23T13:51:38.334 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086.front.sepia.ceph.com 2020-11-23T13:51:38.334 INFO:teuthology.orchestra.run.smithi086:> grep smithi086.front.sepia.ceph.com /etc/hosts 2020-11-23T13:51:38.383 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86 smithi086.front.sepia.ceph.com smithi086 2020-11-23T13:51:38.383 INFO:teuthology.orchestra.run.smithi086:> sudo hostname smithi086 2020-11-23T13:51:38.460 INFO:teuthology.orchestra.run.smithi086:> sudo sed -i -e 's/smithi086.front.sepia.ceph.com/smithi086/g' /etc/hosts 2020-11-23T13:51:38.512 INFO:teuthology.orchestra.run.smithi086:> sudo sed -i -e 's/172.21.15.86/172.21.15.86/g' /etc/hosts 2020-11-23T13:51:38.564 INFO:teuthology.orchestra.run.smithi086:> sudo sed -i -e 's/smithi086.front.sepia.ceph.com/smithi086/g' /etc/hostname 2020-11-23T13:51:38.618 INFO:teuthology.orchestra.run.smithi086:> sudo hostname smithi086 2020-11-23T13:51:38.669 INFO:teuthology.provision.fog.smithi086:Deploy complete! 2020-11-23T13:51:38.669 DEBUG:teuthology.parallel:result is None 2020-11-23T13:51:40.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:51:40.736 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-23T13:51:46.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:51:46.748 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-23T13:51:52.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:51:52.762 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-23T13:51:58.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:51:58.775 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-23T13:52:04.777 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:52:04.869 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-23T13:52:04.869 INFO:teuthology.orchestra.run.smithi169:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-23T13:52:05.270 INFO:teuthology.provision.fog.smithi169:Node is ready 2020-11-23T13:52:05.271 INFO:teuthology.orchestra.run.smithi169:> hostname 2020-11-23T13:52:05.318 INFO:teuthology.orchestra.run.smithi169.stdout:smithi169.front.sepia.ceph.com 2020-11-23T13:52:05.318 INFO:teuthology.orchestra.run.smithi169:> grep smithi169.front.sepia.ceph.com /etc/hosts 2020-11-23T13:52:05.366 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.15.169 smithi169.front.sepia.ceph.com smithi169 2020-11-23T13:52:05.367 INFO:teuthology.orchestra.run.smithi169:> sudo hostname smithi169 2020-11-23T13:52:05.484 INFO:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/smithi169.front.sepia.ceph.com/smithi169/g' /etc/hosts 2020-11-23T13:52:05.498 INFO:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/172.21.15.169/172.21.15.169/g' /etc/hosts 2020-11-23T13:52:05.563 INFO:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/smithi169.front.sepia.ceph.com/smithi169/g' /etc/hostname 2020-11-23T13:52:05.620 INFO:teuthology.orchestra.run.smithi169:> sudo hostname smithi169 2020-11-23T13:52:05.673 INFO:teuthology.provision.fog.smithi169:Deploy complete! 2020-11-23T13:52:05.674 DEBUG:teuthology.parallel:result is None 2020-11-23T13:52:05.803 INFO:teuthology.lock.ops:Checking smithi086.front.sepia.ceph.com 2020-11-23T13:52:05.804 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-23T13:52:05.819 INFO:teuthology.lock.ops:Checking smithi169.front.sepia.ceph.com 2020-11-23T13:52:05.819 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-23T13:52:05.834 INFO:teuthology.lock.ops:Updating [smithi086.front.sepia.ceph.com]: set os type and version on server 2020-11-23T13:52:05.834 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-23T13:52:05.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:52:05.904 INFO:teuthology.orchestra.run.smithi086:> true 2020-11-23T13:52:06.125 INFO:teuthology.orchestra.run.smithi086:> uname -m 2020-11-23T13:52:06.175 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2020-11-23T13:52:06.176 INFO:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2020-11-23T13:52:06.223 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Ubuntu" 2020-11-23T13:52:06.224 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-23T13:52:06.224 INFO:teuthology.orchestra.run.smithi086.stdout:ID=ubuntu 2020-11-23T13:52:06.224 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE=debian 2020-11-23T13:52:06.224 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-23T13:52:06.225 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="18.04" 2020-11-23T13:52:06.225 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-23T13:52:06.225 INFO:teuthology.orchestra.run.smithi086.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-23T13:52:06.225 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-23T13:52:06.225 INFO:teuthology.orchestra.run.smithi086.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-23T13:52:06.226 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_CODENAME=bionic 2020-11-23T13:52:06.226 INFO:teuthology.orchestra.run.smithi086.stdout:UBUNTU_CODENAME=bionic 2020-11-23T13:52:06.227 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2020-11-23T13:52:06.241 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: set os type and version on server 2020-11-23T13:52:06.242 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-23T13:52:06.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:52:06.310 INFO:teuthology.orchestra.run.smithi169:> true 2020-11-23T13:52:06.489 INFO:teuthology.orchestra.run.smithi169:> uname -m 2020-11-23T13:52:06.534 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2020-11-23T13:52:06.534 INFO:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2020-11-23T13:52:06.582 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="Ubuntu" 2020-11-23T13:52:06.583 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-23T13:52:06.583 INFO:teuthology.orchestra.run.smithi169.stdout:ID=ubuntu 2020-11-23T13:52:06.583 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE=debian 2020-11-23T13:52:06.583 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-23T13:52:06.583 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="18.04" 2020-11-23T13:52:06.584 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-23T13:52:06.584 INFO:teuthology.orchestra.run.smithi169.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-23T13:52:06.584 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-23T13:52:06.584 INFO:teuthology.orchestra.run.smithi169.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-23T13:52:06.584 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_CODENAME=bionic 2020-11-23T13:52:06.585 INFO:teuthology.orchestra.run.smithi169.stdout:UBUNTU_CODENAME=bionic 2020-11-23T13:52:06.585 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2020-11-23T13:52:06.600 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2020-11-23T13:52:06.633 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi086.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6XWPP5JBJFtcNewLiaYFp6e++wTUsI6HpJQnXa4yzirBQAv2qfX1bhrBGJCVlOaCwgPJ6ViZD5wC3sOBDOFSOX9F5K5E79ModJ6t2HhOmmT6gOGU0DOtCIW+5b2SZ/R7hLc+4rr9IkRJAdKOmgPypn2U5jq0mn++CeU45jaGXsPR+qp4dHPV7KjMFpuP7h+WENmrPIwUHdP5ya1Nnq+srvqusQmNuWvaVNY38gAeoFUbyFuZKPs9ip63GBKxuP6iQJuW0bF3gH768BVZUrnvJfm7pKN/qJPpIb3A5Y4hjXkAl903SyrGGXnnzlLpJ642TdK0wCBm4JHMERK8EZ4/b smithi169.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCrS9Zba+Tvg+g2wtUeXpYtNCX7Dt3D96Ic788BmVvYHPxytNNYR5myH8PLPn34hgZURaI1mc9fOEcLY/t3u7qBvAkuPP7yBfvOnVK3CgMzmFqfgGvnI1DNbqMmIfG2NjfmAL2m6Hb14wD0C+8+Ox7iHhWlvd9lDicapVZf7P8OzPu2X5gyccq/+gsbPUokO4wiXdd7cO2Up+kEPRzU05nqxILzWnf+4DAhEayzV1qzq3RBQWeSQG1TOZanRe5d9CiUK+mkNIFFQkBYW7iX6Ozphqqq7tySwyw00u4knscChN5sYOFFgw9ZbRzszdgrCjqgdXT6t+8w2/kQUuUGRD0T 2020-11-23T13:52:06.633 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-23T13:52:06.683 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-11-23T13:52:06.717 ERROR:teuthology.timer:Failed to write timing.yaml ! Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/timer.py", line 112, in write yaml.safe_dump(self.data, f, default_flow_style=False) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 28, in represent self.serialize(node) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/serializer.py", line 55, in serialize self.emit(DocumentEndEvent(explicit=self.use_explicit_end)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/emitter.py", line 115, in emit self.state() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/emitter.py", line 220, in expect_document_end self.flush_stream() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/emitter.py", line 790, in flush_stream self.stream.flush() OSError: [Errno 28] No space left on device 2020-11-23T13:52:06.718 INFO:teuthology.task.internal:Saving configuration 2020-11-23T13:52:06.730 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-11-23T13:52:06.739 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-11-23T13:52:06.757 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-23 13:46:40.699562', 'description': '/home/teuthworker/archive/teuthology-2020-11-23_03:05:03-rgw-master-distro-basic-smithi/5647464', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKkR2CK2ztqYjnQ/BJkQ1c2jPLae1kClZ6qCAY4gZDt6', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi086.front.sepia.ceph.com'} 2020-11-23T13:52:06.775 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-23 13:46:40.699585', 'description': '/home/teuthworker/archive/teuthology-2020-11-23_03:05:03-rgw-master-distro-basic-smithi/5647464', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHv/dll/3dqZXNTGib4jINM5HLIij7ApBHor3HsipcCE', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi169.front.sepia.ceph.com'} 2020-11-23T13:52:06.775 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-11-23T13:52:06.806 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2020-11-23T13:52:06.806 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2020-11-23T13:52:06.807 INFO:teuthology.run_tasks:Running task console_log... 2020-11-23T13:52:06.879 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f814b70e8c8>, signals=[15]) 2020-11-23T13:52:06.880 INFO:teuthology.run_tasks:Running task internal.connect... 2020-11-23T13:52:06.887 INFO:teuthology.task.internal:Opening connections... 2020-11-23T13:52:06.888 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2020-11-23T13:52:06.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:52:06.961 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2020-11-23T13:52:06.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-23T13:52:07.032 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-11-23T13:52:07.041 INFO:teuthology.orchestra.run.smithi086:> uname -m 2020-11-23T13:52:07.155 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2020-11-23T13:52:07.156 INFO:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2020-11-23T13:52:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Ubuntu" 2020-11-23T13:52:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-23T13:52:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:ID=ubuntu 2020-11-23T13:52:07.204 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE=debian 2020-11-23T13:52:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-23T13:52:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="18.04" 2020-11-23T13:52:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-23T13:52:07.205 INFO:teuthology.orchestra.run.smithi086.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-23T13:52:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-23T13:52:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-23T13:52:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_CODENAME=bionic 2020-11-23T13:52:07.206 INFO:teuthology.orchestra.run.smithi086.stdout:UBUNTU_CODENAME=bionic 2020-11-23T13:52:07.207 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2020-11-23T13:52:07.222 INFO:teuthology.orchestra.run.smithi169:> uname -m 2020-11-23T13:52:07.228 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2020-11-23T13:52:07.228 INFO:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2020-11-23T13:52:07.275 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="Ubuntu" 2020-11-23T13:52:07.275 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-23T13:52:07.275 INFO:teuthology.orchestra.run.smithi169.stdout:ID=ubuntu 2020-11-23T13:52:07.275 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE=debian 2020-11-23T13:52:07.276 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-23T13:52:07.276 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="18.04" 2020-11-23T13:52:07.276 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-23T13:52:07.276 INFO:teuthology.orchestra.run.smithi169.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-23T13:52:07.276 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-23T13:52:07.277 INFO:teuthology.orchestra.run.smithi169.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-23T13:52:07.277 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_CODENAME=bionic 2020-11-23T13:52:07.277 INFO:teuthology.orchestra.run.smithi169.stdout:UBUNTU_CODENAME=bionic 2020-11-23T13:52:07.277 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2020-11-23T13:52:07.291 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-11-23T13:52:07.336 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 90, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 69, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/__init__.py", line 251, in serialize_remote_roles yaml.safe_dump(info_yaml, info_file, default_flow_style=False) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 28, in represent self.serialize(node) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/serializer.py", line 55, in serialize self.emit(DocumentEndEvent(explicit=self.use_explicit_end)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/emitter.py", line 115, in emit self.state() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/emitter.py", line 220, in expect_document_end self.flush_stream() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/emitter.py", line 790, in flush_stream self.stream.flush() OSError: [Errno 28] No space left on device 2020-11-23T13:52:07.354 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=652cd9e717224c5f8a77ac3408180ef8 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 90, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 69, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/__init__.py", line 251, in serialize_remote_roles yaml.safe_dump(info_yaml, info_file, default_flow_style=False) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 28, in represent self.serialize(node) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/serializer.py", line 55, in serialize self.emit(DocumentEndEvent(explicit=self.use_explicit_end)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/emitter.py", line 115, in emit self.state() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/emitter.py", line 220, in expect_document_end self.flush_stream() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/emitter.py", line 790, in flush_stream self.stream.flush() OSError: [Errno 28] No space left on device 2020-11-23T13:52:07.354 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2020-11-23T13:52:07.449 DEBUG:teuthology.run_tasks:Unwinding manager internal.lock_machines 2020-11-23T13:52:07.495 INFO:teuthology.nuke:Checking targets against current locks 2020-11-23T13:52:07.588 DEBUG:teuthology.nuke:shortname: smithi169 2020-11-23T13:52:07.588 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-11-23T13:52:07.591 DEBUG:teuthology.nuke:shortname: smithi086 2020-11-23T13:52:07.591 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-11-23T13:52:07.606 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-23 13:46:40.699585', 'description': '/home/teuthworker/archive/teuthology-2020-11-23_03:05:03-rgw-master-distro-basic-smithi/5647464', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCrS9Zba+Tvg+g2wtUeXpYtNCX7Dt3D96Ic788BmVvYHPxytNNYR5myH8PLPn34hgZURaI1mc9fOEcLY/t3u7qBvAkuPP7yBfvOnVK3CgMzmFqfgGvnI1DNbqMmIfG2NjfmAL2m6Hb14wD0C+8+Ox7iHhWlvd9lDicapVZf7P8OzPu2X5gyccq/+gsbPUokO4wiXdd7cO2Up+kEPRzU05nqxILzWnf+4DAhEayzV1qzq3RBQWeSQG1TOZanRe5d9CiUK+mkNIFFQkBYW7iX6Ozphqqq7tySwyw00u4knscChN5sYOFFgw9ZbRzszdgrCjqgdXT6t+8w2/kQUuUGRD0T', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi169.front.sepia.ceph.com'} 2020-11-23T13:52:07.608 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-23 13:46:40.699562', 'description': '/home/teuthworker/archive/teuthology-2020-11-23_03:05:03-rgw-master-distro-basic-smithi/5647464', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6XWPP5JBJFtcNewLiaYFp6e++wTUsI6HpJQnXa4yzirBQAv2qfX1bhrBGJCVlOaCwgPJ6ViZD5wC3sOBDOFSOX9F5K5E79ModJ6t2HhOmmT6gOGU0DOtCIW+5b2SZ/R7hLc+4rr9IkRJAdKOmgPypn2U5jq0mn++CeU45jaGXsPR+qp4dHPV7KjMFpuP7h+WENmrPIwUHdP5ya1Nnq+srvqusQmNuWvaVNY38gAeoFUbyFuZKPs9ip63GBKxuP6iQJuW0bF3gH768BVZUrnvJfm7pKN/qJPpIb3A5Y4hjXkAl903SyrGGXnnzlLpJ642TdK0wCBm4JHMERK8EZ4/b', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi086.front.sepia.ceph.com'} 2020-11-23T13:52:07.646 INFO:teuthology.orchestra.console:Power off smithi169 2020-11-23T13:52:07.647 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-23T13:52:07.658 INFO:teuthology.orchestra.console:Power off smithi086 2020-11-23T13:52:07.658 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-23T13:52:07.666 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-23T13:52:07.677 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-23T13:52:11.685 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-23T13:52:11.697 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status